#linuxcnc-devel Logs

Jul 03 2020

06:34 PM skunkworks: so.. If you have an eoffset ratio in the ini - that is in effect if the eoffset is enabled or not?
06:38 PM skunkworks: It isn't an ini. setable pin either..
06:38 PM skunkworks: I guess I could change the ini.x_mumble_acceleration / vel when needed.
06:41 PM skunkworks: for a lathe - I could just used the offset componant... IT doesn't look like anything anyway (compared to the 2 axis tool motion)
07:43 PM andypugh: I am ready to give up on RTAI. I have expended all my spare time on it for a cpuple of weeks and made no prigress at all.
07:46 PM memfrob: I haven't had a chance to look at the RTAI code yet, andypugh
07:47 PM memfrob: I was going to try a few things out but I'm still in the middle of rebuilding my Linux servers, it's hot in here so I haven't had the motivation.
07:49 PM andypugh: I thought we were on to something with the netconsole logging and the apparently unambiguous pointer to an fxrstor64 in rtai_switch_task. But, actualyly, I see no sign that that code _ever_ runs. I think that the crash dump info is bogus.
07:50 PM andypugh: But without that, we are left with nothing.
08:02 PM andypugh: memfrob: I will say that I am becoming a bit suspicious of “Streamer” which runs in some of the tests, but not, typically, in real machines. I suspect it of hitting switch_rtai_task raher hard. But I don’t currently have a working system to investigate this with.
08:06 PM CaptHindsight: more AC coming here next week
08:06 PM CaptHindsight: UPS decided to not deliver here today
08:07 PM CaptHindsight: kept our AC in a trailer 30 miles away
08:16 PM memfrob: andypugh, for debugging purposes, would not fortifying the RTAI source _possibly_ ease debugging?
08:17 PM memfrob: The _FORITFY_SOURCE stuff is no longer needed now that math is grabbed from musl
08:17 PM andypugh: I have no idea what Fortify Source does. Perhaps you are mistaking me for a computer programmer?
08:18 PM memfrob: https://access.redhat.com/blogs/766093/posts/1976213 -- the value is set to 2 right now
08:18 PM andypugh: (I did manage, eventually, to work out where in the RTAI tree to put the -g switch to get the source in the objdump)
08:18 PM memfrob: "With _FORTIFY_SOURCE set to 2 some more checking is added, but some conforming programs might fail. Some of the checks can be performed at compile time, and result in compiler warnings; other checks take place at run time, and result in a run-time error if the check fails."
08:19 PM memfrob: RTAI_USER_CFLAGS
08:21 PM andypugh: I ended up putting it in a sub makefile (but not a submakefile) in base/sched
08:23 PM memfrob: Only part that switch happens is.. if ((!new_task->lnxtsk || !rt_current->lnxtsk))
08:24 PM jepler: when you say "streamer" do you mean the realtime component you get with "loadrt streamer"? I don't see that it does anything particularly interesting, it calls some HAL APIs that are not used in a lot of cases, but they boil down to nothing more than memory accesses.
08:25 PM andypugh: jepler: You are probably right, and I am probably barking up one more wrong tree in this forest of wrong trees
08:26 PM memfrob: Since this crash happens on everything, has jepler taken a look at this?
08:26 PM jepler: FORTIFY SOURCE has never been particularly well documented, but it is only appropriate for things that are in userspace. if it is affecting the build of kernel modules, then they are incorrectly including headers intended for userspace.
08:26 PM memfrob: I will remove those then since they're not needed.
08:26 PM memfrob: RTAI previously wouldn't compile with it, but all math is replaced now.
08:27 PM memfrob: *wouldn't compile _without_ it
08:28 PM memfrob: Any changes to RTAI is going to take a bit though.
08:28 PM memfrob: (for there to be an actual commit)
08:29 PM jepler: I don't have spare cycles for debugging rtai
08:30 PM memfrob: andypugh, you can rip out all mentions of FORTIFY_SOURCE in configure.ac and see if your stacktrace changes.
08:31 PM andypugh: memfrob: Which part of “I give up” was unclear?
08:31 PM memfrob: I wasn't here for that, didn't read back log.
08:32 PM andypugh: I am very grateful for all your efforts, over the years, but frankly we are getting nowhere.
08:32 PM memfrob: RTAI has been a mess and I've done my best to try and maintain it.
08:32 PM andypugh: jepler: Maybe I mean sampler, Maybe I have no ide what I mean.
08:32 PM andypugh: https://paste.ubuntu.com/p/S6WZRPzDrH/
08:33 PM memfrob: gstreamer ;)
08:33 PM andypugh: It would help a lot if when dmesg said “[10629.037569] finit_module: fd=3, uargs=0000000052648593, flags=0” it told you _which_ module it just loaded
08:35 PM jepler: I am _guessing_ it is the module X named below in "X uses Y!" messages
08:35 PM memfrob: I think there is an option for that
08:35 PM andypugh: .I _might_ go back to 4.14.174 and the NTULINUX RTAI and try not running the tests that use sampler and/or streamer to see what happens.
08:36 PM jepler: so in that test you're running something with sampler & streamer & stepgen. and you apparently see "a lot" of "switch_rtai_tasks"
08:36 PM jepler: maybe you're seeing that from tests that have two threads? Most if not all stepgen will have 2 threads
08:36 PM memfrob: config DEBUG_DRIVER I know prints all the "driver" output
08:36 PM jepler: otherwise, with just 1 thread, there is never a time to switch from one RT thread to another
08:37 PM andypugh: As, frankly, jumoing through Paolo’s hoops to build his version of RTAI has just led to frustration and no progress.
08:37 PM memfrob: I will try Paolo's code as-is as well.
08:38 PM andypugh: jepler: Well, the line that the crash dunp always pointed me to looked like it only ever ran when switching between two FPU tasks.
08:38 PM memfrob: Building his code involves some kernel Kconfig knowledge and C library mucking about.
08:39 PM jepler: I applaud and am grateful to both of you for the time you've spent on this
08:39 PM memfrob: I can _do_ all of this, but I need a dev system back.
08:39 PM jepler: normally in linuxcnc the "slow" thread would have FP and the "fast" thread would be intended to not have FP .. except who knows about such things
08:40 PM memfrob: andypugh, I understand you gave up, but if I make progress with this, will you check into some things for me?
08:40 PM jepler: I mean, maybe we're sending in the wrong flag, or maybe the flag has to be ignored (e.g., because use of the XMM registers will be added even when "floating point math" isn't being done, like we saw much earlier in this story of trying to get LinuxCNC working together with modern RTAI, and this is really about saving/restoring XMM registers)
08:40 PM andypugh: Don’t get me wrong, Paolo has been helpful and has even compiled and run LinuxCNC on his own system. And I fully understand that he wanted us to be looking at the same code base (I would not take kindly to somone giving me a hard time about MachineKit bugs)
08:40 PM memfrob: I won't say anything for about 2 weeks at this rate.
08:40 PM memfrob: did he run into this crash?
08:42 PM jepler: Do either of you know whether Paolo says whether "does / does not crash" type tests can be run in VMs with the official RTAI?
08:42 PM andypugh: memfrob: a) Yes, I am happy to test stuff, but I can’t let RTAI get in the way of the 2.8 release any longer.
08:42 PM andypugh: b) Yes, he saw the LinuxCNC crashes _and_ the RTAI-only crashes,
08:42 PM jepler: i.e., would upstream act if provided with a program that I say crashes, but I only tested in a VM?
08:44 PM andypugh: jepler: I don’t know. But if I was him I would want to see a test-case on a real machine. This used to look hard, but now he does habe LinuxCNC.
08:45 PM memfrob: If LinuxCNC isn't going to support RTAI 5 then you're still going to have RTAI 5 references in the 2.8 branch
08:45 PM memfrob: LinuxCNC 2.8 release I mean
08:45 PM andypugh: Paolo is definitely bothered about the load/unload crash, but not _very_ worried as it is a rather artificial test.
08:45 PM memfrob: So wait, it is crashing for him?
08:46 PM andypugh: Yes, both LInuxCNC test cases and the rtai load/unload.
08:46 PM jepler: andypugh: which kernel version have you concentrated your testing on?
08:47 PM memfrob: Well, it's nothing I did then!
08:47 PM jepler: 4.19 ?
08:47 PM memfrob: That's a relief for me haha.
08:48 PM andypugh: 4.19.114
08:49 PM andypugh: Thiough, to be fully Paolo complaint you would use 4.19.114-cip24
08:49 PM andypugh: (The RTAI patch is for 4.19.114-cip24 bit takes cleanly on plain 4.19.114)
08:51 PM andypugh: But I found that, for me, 4.19.114 would kernel panic with the official RTAI kern/latency test. But not with the vanilla 4.19.114
08:52 PM andypugh: But, now, kern/latecny kernel panics my machine even with my previously OK kernel after a fresh SVN pull or RTAI. I suspect I lost a tweak I made a few weeks ago.
08:54 PM andypugh: So, if I _do_ go back to RTAI it will be the NTULINUX one for 4.14.174. If only because we can make deb packages of that. Vulcano doesn’t do packages as far as I know.
08:58 PM andypugh: I was hoping that by talking to the RTAI folk I would get more people looking at the problem, but I only actually got Paolo and he just gave a lot of work to do to match his setup and no actual help with the problem (though I am ot saying he didn’t try)
10:27 PM memfrob: Paolo and I are the only people working on RTAI.
10:28 PM memfrob: Rest are users lol