#linuxcnc-devel Logs

Apr 24 2020

01:07 AM linuxcnc-build: build #5063 of 1404.rip-wheezy-rtpreempt-amd64 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1404.rip-wheezy-rtpreempt-amd64/builds/5063 blamelist: Phillip Carter <phillcarter54@gmail.com>
01:11 AM linuxcnc-build: build #4859 of 1400.rip-wheezy-i386 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1400.rip-wheezy-i386/builds/4859 blamelist: Phillip Carter <phillcarter54@gmail.com>
01:11 AM linuxcnc-build: build #4860 of 1403.rip-wheezy-amd64 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1403.rip-wheezy-amd64/builds/4860 blamelist: Phillip Carter <phillcarter54@gmail.com>
02:06 AM linuxcnc-build: build #1716 of 1630.rip-stretch-rtpreempt-amd64 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1630.rip-stretch-rtpreempt-amd64/builds/1716 blamelist: Phillip Carter <phillcarter54@gmail.com>
02:12 AM linuxcnc-build: build #4531 of 1401.rip-wheezy-rtai-i386 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1401.rip-wheezy-rtai-i386/builds/4531 blamelist: Phillip Carter <phillcarter54@gmail.com>
02:18 AM linuxcnc-build: build #3329 of 1530.rip-jessie-rtpreempt-amd64 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1530.rip-jessie-rtpreempt-amd64/builds/3329 blamelist: Phillip Carter <phillcarter54@gmail.com>
02:20 AM linuxcnc-build: build #6720 of 0000.checkin is complete: Failure [4failed] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/0000.checkin/builds/6720 blamelist: Phillip Carter <phillcarter54@gmail.com>
02:51 AM linuxcnc-build: build #5920 of 1301.rip-precise-rtai-i386 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1301.rip-precise-rtai-i386/builds/5920 blamelist: Phillip Carter <phillcarter54@gmail.com>
02:57 AM linuxcnc-build: build #3327 of 1500.rip-jessie-i386 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1500.rip-jessie-i386/builds/3327 blamelist: Phillip Carter <phillcarter54@gmail.com>
03:07 AM linuxcnc-build: build #6721 of 0000.checkin is complete: Failure [4failed] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/0000.checkin/builds/6721 blamelist: Phillip Carter <phillcarter54@gmail.com>
08:36 AM jepler: what the heck. One of my two pi4 systems doesn't write a ".deb" file when I "mk-build-deps". Tested on linuxcnc package and hello package. The other seems to work.
08:39 AM linuxcnc-build: build #4862 of 1400.rip-wheezy-i386 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1400.rip-wheezy-i386/builds/4862 blamelist: andypugh <andy@bodgesoc.org>
08:39 AM linuxcnc-build: build #4863 of 1403.rip-wheezy-amd64 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1403.rip-wheezy-amd64/builds/4863 blamelist: andypugh <andy@bodgesoc.org>
08:41 AM linuxcnc-build: build #5922 of 1301.rip-precise-rtai-i386 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1301.rip-precise-rtai-i386/builds/5922 blamelist: andypugh <andy@bodgesoc.org>
08:48 AM jepler: anyway, looks like somewhere above 10% of the time I can reproduce the failure in tests/tool-info/random-with-startup-tool/ on my pi4
08:57 AM linuxcnc-build: build #5066 of 1404.rip-wheezy-rtpreempt-amd64 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1404.rip-wheezy-rtpreempt-amd64/builds/5066 blamelist: andypugh <andy@bodgesoc.org>
08:58 AM linuxcnc-build: build #4382 of 1402.rip-wheezy-rtpreempt-i386 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1402.rip-wheezy-rtpreempt-i386/builds/4382 blamelist: andypugh <andy@bodgesoc.org>
09:40 AM jepler: 13 of 100 runs failed
09:41 AM jepler: adding a 1 second sleep just to see what happens
09:41 AM jepler: it takes longer to run now
09:42 AM dgarr: i have some work in dgarr/mtest, pretty sure userspace delays cause these, for example set [TASK]CYCLE_TIME=1.0
09:44 AM linuxcnc-build: build #3329 of 1500.rip-jessie-i386 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1500.rip-jessie-i386/builds/3329 blamelist: andypugh <andy@bodgesoc.org>
09:49 AM jepler: dgarr: I can believe it
09:49 AM jepler: I think there's a good chance that enlarging the tool table allowed it to happen more frequently
09:50 AM jepler: I doubt we can guarantee that the tool information will be in the status buffer with any particular deadline
09:57 AM jepler: dgarr: the general approach you took in verify_status_buffer sounds good to me
09:57 AM linuxcnc-build: build #6723 of 0000.checkin is complete: Failure [4failed] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/0000.checkin/builds/6723 blamelist: andypugh <andy@bodgesoc.org>
09:57 AM jepler: I mean I'm not thrilled to have to add it, but a variable delay is better than the fixed delay I added. Assuming it works
10:01 AM jepler: I'm less certain about ignoring lines in startup-gcode-abort, but having moves and pauses in the rs274 startup gcode is pretty crazy in my book
10:05 AM jepler: In fact, I think that startup-gcode-abort test should be left as-is with its skip file as there's an open bug related to it https://github.com/linuxcnc/linuxcnc/issues/49
10:43 AM linuxcnc-build: build #6705 of 1306.rip-precise-amd64 is complete: Failure [4failed compile] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1306.rip-precise-amd64/builds/6705 blamelist: Jeff Epler <jepler@gmail.com>
10:43 AM linuxcnc-build: build #6702 of 1300.rip-precise-i386 is complete: Failure [4failed compile] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1300.rip-precise-i386/builds/6702 blamelist: Jeff Epler <jepler@gmail.com>
10:43 AM linuxcnc-build: build #5923 of 1301.rip-precise-rtai-i386 is complete: Failure [4failed compile] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1301.rip-precise-rtai-i386/builds/5923 blamelist: Jeff Epler <jepler@gmail.com>
11:14 AM linuxcnc-build: build #4864 of 1403.rip-wheezy-amd64 is complete: Failure [4failed runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1403.rip-wheezy-amd64/builds/4864 blamelist: Jeff Epler <jepler@gmail.com>
11:19 AM linuxcnc-build: build #326 of 1640.rip-buster-rtpreempt-amd64 is complete: Failure [4failed runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1640.rip-buster-rtpreempt-amd64/builds/326 blamelist: Jeff Epler <jepler@gmail.com>
11:26 AM linuxcnc-build: build #6724 of 0000.checkin is complete: Failure [4failed] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/0000.checkin/builds/6724 blamelist: Jeff Epler <jepler@gmail.com>
11:28 AM rene_dev_: jepler I think pulling toolinfo out of the buffer isnt hard
11:28 AM rene_dev_: it doesnt belong there anyway
11:29 AM rene_dev_: does it affect normal operation, or only the test
11:30 AM linuxcnc-build: build #6706 of 1306.rip-precise-amd64 is complete: Failure [4failed compile] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1306.rip-precise-amd64/builds/6706 blamelist: Jeff Epler <jepler@gmail.com>
11:30 AM linuxcnc-build: build #5924 of 1301.rip-precise-rtai-i386 is complete: Failure [4failed compile] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1301.rip-precise-rtai-i386/builds/5924 blamelist: Jeff Epler <jepler@gmail.com>
11:30 AM linuxcnc-build: build #6703 of 1300.rip-precise-i386 is complete: Failure [4failed compile] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1300.rip-precise-i386/builds/6703 blamelist: Jeff Epler <jepler@gmail.com>
11:30 AM rene_dev_: I will take care of the pyhton test PRs tonight
11:30 AM jepler: rene_dev_: okay, I'll hand these intermittent failures off to you. let me know if you want to hand it back or have a PR I could look at
11:47 AM jepler: we'll call this way of solving problems "asking for the buck"
12:12 PM linuxcnc-build: build #6725 of 0000.checkin is complete: Failure [4failed] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/0000.checkin/builds/6725 blamelist: Jeff Epler <jepler@gmail.com>
01:27 PM seb_kuzminsky: andypugh: this test script crashes my buster rtai machine in <5 minutes usually: http://highlab.com/~seb/linuxcnc/lots-of-tests
01:28 PM seb_kuzminsky: if i uncomment the lines at the bottom of the for loop, it crashes much less often
01:36 PM linuxcnc-build: build #6705 of 1300.rip-precise-i386 is complete: Failure [4failed compile] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1300.rip-precise-i386/builds/6705 blamelist: Jeff Epler <jepler@gmail.com>, andypugh <andy@bodgesoc.org>
01:36 PM linuxcnc-build: build #6708 of 1306.rip-precise-amd64 is complete: Failure [4failed compile] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1306.rip-precise-amd64/builds/6708 blamelist: Jeff Epler <jepler@gmail.com>, andypugh <andy@bodgesoc.org>
01:42 PM linuxcnc-build: build #5926 of 1301.rip-precise-rtai-i386 is complete: Failure [4failed compile] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1301.rip-precise-rtai-i386/builds/5926 blamelist: Jeff Epler <jepler@gmail.com>, andypugh <andy@bodgesoc.org>
02:06 PM linuxcnc-build: build #4866 of 1400.rip-wheezy-i386 is complete: Failure [4failed runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1400.rip-wheezy-i386/builds/4866 blamelist: Jeff Epler <jepler@gmail.com>, andypugh <andy@bodgesoc.org>
02:06 PM andypugh: seb_kuzminsky: Thanks. I will try it.
02:07 PM seb_kuzminsky: i've seen it crash after less than 50 iterations, so i don't think it's a memory leak, but rather a race condition somewhere
02:07 PM andypugh: 50,000 cycles over 6 hours of just loading and stopping realtime didn’t crash it. And 10 hours running LinuxCNC overnight was OK too.
02:08 PM seb_kuzminsky: i didn't try the 'realtime start; realtime stop' test, but i did leave it up running linuxcnc overnight and it didn't crash for me eithe
02:08 PM seb_kuzminsky: maybe it's starting & stopping hal? or loading and unloading modules?
02:09 PM andypugh: So, annoyingly, it is probably fit for purpose, but not for testing to prove that,
02:10 PM seb_kuzminsky: i'd argue it's not reliably fit for purpose, if there's a 1% or 0.1% chance that it'll panic the machine when you quit linuxcnc
02:12 PM andypugh: It just managed 150 iterations of your script.
02:12 PM andypugh: (then crashed)
02:13 PM seb_kuzminsky: that's about what i get too - though there's some kind of distribution, with a peak around 100-200 iterations or so
02:15 PM andypugh: Does it completely crash your machine?
02:16 PM andypugh: I just got a final: RTAI[sched]: timer setup = 109 ns, resched latency = 0 ns.
02:16 PM seb_kuzminsky: yes, kernel panic and complete lockup
02:16 PM andypugh: So it (maybe) crashed during the Usermode check?
02:16 PM seb_kuzminsky: i have some phone pics of the screen, hang on
02:16 PM andypugh: I don’t see the kernel panic.
02:17 PM andypugh: But I am working via ssh
02:17 PM seb_kuzminsky: it only shows up on the console
02:17 PM seb_kuzminsky: it's too quick and disastrous to get the message out to the log files
02:18 PM andypugh: So no ip to do addr2line with?
02:21 PM seb_kuzminsky: http://highlab.com/~seb/linuxcnc/rtai-crash.2020-04-24/
02:22 PM seb_kuzminsky: this one triggered a kernel BUG check: http://highlab.com/~seb/linuxcnc/rtai-crash.2020-04-24/rtai-crash-IMG_20200424_115035.jpg
02:22 PM seb_kuzminsky: butif i was a gambling man i bet the problem was memory corruption long before we came to that line
02:22 PM seb_kuzminsky: afk for a bit
02:51 PM -!- #linuxcnc-devel mode set to +v by ChanServ
03:25 PM -!- #linuxcnc-devel mode set to +v by ChanServ
03:25 PM jepler: andypugh: oops, turns out that pull request of mine wasn't ready :( I'll follow up soon
03:25 PM andypugh: Ah well
03:25 PM andypugh: Do you have an opinion on the bazillion pull requests from Damian?
03:26 PM -!- #linuxcnc-devel mode set to +v by ChanServ
03:26 PM jepler: I have not looked. I know the ones on mesaflash looked good at least superficially.
03:27 PM jepler: are there also ones on linuxcnc?
03:28 PM jepler: looks like a mix of changing #!-lines in a way that may be required by some other OS and changing things to be a little bit more python3 compliant. I just looked at one.
03:30 PM jepler: so since there's a python2/3 component to this, I'd also bounce it off people who have various python3 stuff cooking, like dewey and rene I guess
03:30 PM rene_dev_: jepler they mostly look good
03:32 PM andypugh: jepler: Are you in a position to sort out the Makefile merge conflict betwixt 2.8 and Master?
03:32 PM andypugh: (I didn’t even look at it, I confess)
03:32 PM jepler: andypugh: If it's the one I created, it's fairly trivial .. but I will wait until 2.8 settles down before resolving it
03:33 PM jepler: looks like highlab/buildbot are taking a break from the internet, :(
03:33 PM andypugh: What is unsettled about 2.8?
03:34 PM jepler: > build #5926 of 1301.rip-precise-rtai-i386 is complete: Failure [failed compile] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1301.rip-precise-rtai-i386/builds/5926 blamelist: Jeff Epler <jepler@gmail.com>, andypugh <andy@bodgesoc.org>
03:34 PM andypugh: Damian’s PRs are all against master. (probably shouldn’t be)
03:35 PM jepler: andypugh: there was a typo in my pull request and it breaks building on the oldest platforms, maybe just precise
03:35 PM jepler: a missing "-"
03:55 PM rene_dev_: jepler andypugh I merged damians PRs
03:55 PM rene_dev_: jepler there are 4 left Im not sure about, can you have a look?
03:59 PM -!- #linuxcnc-devel mode set to +v by ChanServ
04:01 PM -!- #linuxcnc-devel mode set to +v by ChanServ
04:11 PM -!- #linuxcnc-devel mode set to +v by ChanServ
04:15 PM -!- #linuxcnc-devel mode set to +v by ChanServ
04:20 PM rene_dev_: The tests now pass when python is python 3
04:20 PM andypugh: Woot!
04:20 PM andypugh: Do any GUIs work?
04:21 PM andypugh: Kind of apporpriate as today was the final, no, really, this time we mean it, Python2 release.
04:26 PM Tom_L: getting closer every day...
04:33 PM sync: andypugh: axis works fine
04:35 PM rene_dev_: axis works, most other UIs are broken. not all can be fixed
04:36 PM andypugh: PyGTK?
04:38 PM rene_dev_: gtk2 is the problem
04:41 PM -!- #linuxcnc-devel mode set to +v by ChanServ
04:46 PM jepler: Is there a path forward? Does it have to be rewritten in gtk3?
04:47 PM jepler: that is, does each gtk2 UI have to be rewritten
04:48 PM Tom_L: that one grapic lib wasn't included in buster
04:48 PM Tom_L: not a helpful description but i forgot what it was
04:49 PM andypugh: There is a bit of a dodge with gi.repository in many cases.
04:49 PM andypugh: For example, if you need rsvg, then instead of
04:49 PM andypugh: import rsvg
04:49 PM andypugh: you can use
04:50 PM rene_dev_: gtk3 seems to be very broken, I think qt5 is the way to go. there are several good UIs in qt5
04:50 PM andypugh: from gi.repository import Rsvg as rsvg
04:51 PM andypugh: rene_dev_: It’s all well anf good looking for the technically correct solution, but there are a lot of GladeVCPs out there.
04:52 PM sync: yes, but I think that is unfortunately their™ problem :/
04:52 PM rene_dev_: I know :(
04:53 PM rene_dev_: but looking at python3 bindings for gtk2, just doesnt look promising
04:54 PM andypugh: rene_dev_: have you looked at gi.repository?
04:55 PM rene_dev_: yes, I have not gotten far into UI stuff yet. there is more to it ;D
04:55 PM andypugh: So, write a script to convert Glade UI files to QT, How hard van it be?
04:55 PM rene_dev_: oh boy
04:56 PM andypugh: (I am only partially joking, I think wha got JA finally merged was the auto-conversion script)
04:56 PM rene_dev_: speaking of which, there is a issue I have with JA
04:57 PM andypugh: We are in the awkward position that a lot of our users are using machines that they did not, themselves, configure.
04:58 PM andypugh: Of course, we could simply tell them not to update.
04:58 PM rene_dev_: if they didnt configure them, why would they update?
04:59 PM rene_dev_: what annoys me about JA is that with trivkins you still need to set min/max/minvel/maxvel for joints and axis, which is annoying and easy to get wrong for newbies
05:00 PM andypugh: Newbies use stepconf.
05:00 PM rene_dev_: would be cool if the joints could just default to the axis limits, if there is no joint in the ini
05:00 PM Tom_L: andypugh, as far as stepconf will take them
05:01 PM andypugh: rene_dev_: Should be possible.
05:02 PM rene_dev_: https://github.com/LinuxCNC/linuxcnc/blob/master/configs/sim/axis/axis.ini#L146-L163
05:02 PM rene_dev_: other way round, the axis default to the joint limits
05:03 PM rene_dev_: just looking at the example shows the problem, completley diffetent values for joint and axis :)
05:03 PM rene_dev_: look at the acceleration and velocity
05:11 PM andypugh: That does seem a little odd.
05:12 PM andypugh: Some of them show signs of being run through the conversion script. That one doesn’t.
09:00 PM memfrob: I just saw the emails from andypugh, looks like your build bot is crashing?
09:01 PM memfrob: seb_kuzminsky, does the system crash on your personal computer or just the build bot? Andypugh isn't experiencing your problem.
09:10 PM skunkworks: I think it is doing it on real hardware too.. You would have to read back
09:25 PM memfrob: I always have a hard time finding the logs, can someone please just add this to the sticky? http://tom-itx.no-ip.biz:81/~tom-itx/irc/logs/%23linuxcnc-devel/index.html
09:25 PM memfrob: /topic whatever
09:27 PM memfrob: The trace seems to be related to rt2x00usb
09:29 PM memfrob: Do I have to blacklist that module from Kconfig now too?
09:29 PM memfrob: Ah: http://highlab.com/~seb/linuxcnc/rtai-crash.2020-04-24/rtai-crash-IMG_20200424_122222.jpg
09:31 PM memfrob: skbuff.c ..
09:33 PM memfrob: I'm hunting down gentoo bugs right now so I don't have much time but nothing stands out to me at this moment.
09:34 PM memfrob: I remove the usermode check without a problem if that's where you guys think the problem is.
09:36 PM memfrob: along with all the code related to the usermode check. If you think it's worth a shot, email me and I'll write a patch. If it still fails, then I'll need to contact Paolo.
09:37 PM memfrob: Take care all I'm done with Linux for the day.