#linuxcnc-devel Logs

Oct 31 2023

12:36 AM linuxcnc-build: build #3563 of 1660.rip-buster-python3 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1660.rip-buster-python3/builds/3563 blamelist: Rene Hopf <renehopf@mac.com>, Moses McKnight <moses@mcktex.com>
01:03 AM linuxcnc-build: build #10383 of 0000.checkin is complete: Failure [4failed] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/0000.checkin/builds/10383 blamelist: Rene Hopf <renehopf@mac.com>, Moses McKnight <moses@mcktex.com>
01:34 AM linuxcnc-build2: Build [#1781](http://buildbot2.highlab.com/buildbot/#builders/7/builds/1781) of `10-rip.debian-11-bullseye-rtpreempt-amd64` 4failed.
02:11 AM linuxcnc-build2: Build [#1787](http://buildbot2.highlab.com/buildbot/#builders/11/builds/1787) of `10-rip.debian-12-bookworm-rtpreempt-amd64` 4failed.
02:13 AM linuxcnc-build2: Build [#1861](http://buildbot2.highlab.com/buildbot/#builders/13/builds/1861) of `00-checkin` 4failed.
02:40 AM linuxcnc-build: build #3966 of 1640.rip-buster-rtpreempt-amd64 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1640.rip-buster-rtpreempt-amd64/builds/3966 blamelist: Petter Reinholdtsen <pere@hungry.com>
02:46 AM linuxcnc-build2: Build [#1782](http://buildbot2.highlab.com/buildbot/#builders/7/builds/1782) of `10-rip.debian-11-bullseye-rtpreempt-amd64` 8completed with warnings.
02:47 AM pere: something is wrong with RIP in master, see <URL: https://github.com/LinuxCNC/linuxcnc/commit/69207fffd5a3c6d178aa738a24db442c56d79a4a >, test never completed...
02:51 AM linuxcnc-build2: Build [#1788](http://buildbot2.highlab.com/buildbot/#builders/11/builds/1788) of `10-rip.debian-12-bookworm-rtpreempt-amd64` 8completed with warnings.
03:00 AM linuxcnc-build: build #10384 of 0000.checkin is complete: Failure [4failed] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/0000.checkin/builds/10384 blamelist: Petter Reinholdtsen <pere@hungry.com>
03:03 AM linuxcnc-build2: Build [#1862](http://buildbot2.highlab.com/buildbot/#builders/13/builds/1862) of `00-checkin` 3completed successfully.
04:23 AM linuxcnc-build2: Build [#1789](http://buildbot2.highlab.com/buildbot/#builders/11/builds/1789) of `10-rip.debian-12-bookworm-rtpreempt-amd64` 4failed.
04:25 AM linuxcnc-build2: Build [#1863](http://buildbot2.highlab.com/buildbot/#builders/13/builds/1863) of `00-checkin` 4failed.
05:12 AM linuxcnc-build2: Build [#1783](http://buildbot2.highlab.com/buildbot/#builders/14/builds/1783) of `10-rip.debian-10-buster-rtpreempt-amd64` 4failed.
05:13 AM linuxcnc-build2: Build [#1784](http://buildbot2.highlab.com/buildbot/#builders/7/builds/1784) of `10-rip.debian-11-bullseye-rtpreempt-amd64` 4failed.
05:16 AM linuxcnc-build2: Build [#1790](http://buildbot2.highlab.com/buildbot/#builders/11/builds/1790) of `10-rip.debian-12-bookworm-rtpreempt-amd64` 8completed with warnings.
05:50 AM linuxcnc-build2: Build [#1784](http://buildbot2.highlab.com/buildbot/#builders/14/builds/1784) of `10-rip.debian-10-buster-rtpreempt-amd64` 8completed with warnings.
06:05 AM pere: six hour CI run before timing out. Anyone know what is going on? <URL: https://github.com/LinuxCNC/linuxcnc/actions/runs/6700961718/job/18207703611 >
07:08 AM rigid: pere: how do you know it has timed out?
07:12 AM pere: because of the time taken, the message at the end of the log and the last message. if it actually failed, the message would be different.
07:13 AM rigid: it could also exhaust other resources (too many spawned processes or RAM exhaustion)
07:13 AM pere: I assume the CI jobs are allowed to run for 6 hours.
07:13 AM pere: perhaps, but I suspect any resources would run out before six hours had passed.
07:18 AM rigid: pere: maybe run the G0 test with -v and see what happens
07:19 AM rigid: ...or where it halts
07:21 AM pere: seem like a good next step, yes. hope to get there after work.
08:36 AM linuxcnc-build2: Build [#1785](http://buildbot2.highlab.com/buildbot/#builders/14/builds/1785) of `10-rip.debian-10-buster-rtpreempt-amd64` 4failed.
08:37 AM linuxcnc-build2: Build [#1786](http://buildbot2.highlab.com/buildbot/#builders/7/builds/1786) of `10-rip.debian-11-bullseye-rtpreempt-amd64` 8completed with warnings.
09:13 AM linuxcnc-build2: Build [#1786](http://buildbot2.highlab.com/buildbot/#builders/14/builds/1786) of `10-rip.debian-10-buster-rtpreempt-amd64` 8completed with warnings.
09:15 AM linuxcnc-build2: Build [#1787](http://buildbot2.highlab.com/buildbot/#builders/7/builds/1787) of `10-rip.debian-11-bullseye-rtpreempt-amd64` 4failed.
11:11 AM Unterhaus_ is now known as unterhausen
11:55 AM linuxcnc-build2: Build [#1788](http://buildbot2.highlab.com/buildbot/#builders/7/builds/1788) of `10-rip.debian-11-bullseye-rtpreempt-amd64` 8completed with warnings.
12:35 PM linuxcnc-build2: Build [#1868](http://buildbot2.highlab.com/buildbot/#builders/13/builds/1868) of `00-checkin` 3completed successfully.
01:47 PM rmu: does anybody here know gnu parallel? maybe that could be used to speed up test runs.
03:49 PM rigid: rmu: it could but you can't run too many jobs in parallel in github actions it seems
03:49 PM rigid: my first thought was a minimal version that doesn't need all those dependencies, not build docs etc. so the full version needs only to run once
03:50 PM rigid: that way, stuff runs pretty quick here on my box
03:51 PM rigid: i'd also check for sleeps
03:52 PM rigid: also i'm not sure what happens when multiple linuxcnc instances run in parallel on the same host without any sandboxing.
03:53 PM rigid: gnu parallel doesn't do that. just forks processes.
04:35 PM rmu: rigid: tests are also run on developer machines and on buildbot
04:36 PM rmu: building is not a problem, full build takes <15s
04:39 PM rmu: and i think i took care of parallel linuxcnc instances, there is draft PR on github. needs some cleaning though
04:48 PM rigid: rmu: yeah build, but dependency install takes a huge amount of time. I stripped that down significantly. didn't benchmark tho
04:49 PM rigid: btw. I looked into flatbuf and I wonder if we just talk about NML or actual RCS
04:50 PM rigid: since the whole "real-time control system" provides a (shared) memory buffer that can be used by the app totally transparently and I wonder how much work that would be to implement with zeromq+flatbuffers
04:54 PM rigid: looking at the code it seems there was some misconception about (locked) buffer based messaging and packet based messaging. zeromq uses "atomic messages" which are packets
04:56 PM rigid: like for example one can update the state of multiple protocol objects with a single message and the other needs one message per object (attribute)
05:02 PM linuxcnc-build2: Build [#1788](http://buildbot2.highlab.com/buildbot/#builders/14/builds/1788) of `10-rip.debian-10-buster-rtpreempt-amd64` 4failed.
05:03 PM linuxcnc-build2: Build [#1789](http://buildbot2.highlab.com/buildbot/#builders/7/builds/1789) of `10-rip.debian-11-bullseye-rtpreempt-amd64` 4failed.
05:09 PM linuxcnc-build2: Build [#1869](http://buildbot2.highlab.com/buildbot/#builders/13/builds/1869) of `00-checkin` 4failed.
05:15 PM linuxcnc-build: build #3566 of 1660.rip-buster-python3 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1660.rip-buster-python3/builds/3566 blamelist: petterreinholdtsen <pere-github@hungry.com>, Hosted Weblate <hosted@weblate.org>
05:25 PM linuxcnc-build: build #3968 of 1640.rip-buster-rtpreempt-amd64 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1640.rip-buster-rtpreempt-amd64/builds/3968 blamelist: petterreinholdtsen <pere-github@hungry.com>, Hosted Weblate <hosted@weblate.org>
05:40 PM linuxcnc-build: build #10386 of 0000.checkin is complete: Failure [4failed] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/0000.checkin/builds/10386 blamelist: petterreinholdtsen <pere-github@hungry.com>, Hosted Weblate <hosted@weblate.org>
06:04 PM rene-dev5: rigid even with only one core it will run significantly faster, as so much time is spend waiting
06:04 PM rene-dev5: rmu added a environment variable to offset the keys
06:12 PM rene-dev5: rigid which rcs stuff are you talking about
06:17 PM rigid: rene-dev5: everything that handles message handling in the background so the application just handles a buffer that appears to be a normal local buffer. (passing, handling, locking, etc.)
06:19 PM rigid: offset the keys? instead of using different buffers? hm
06:21 PM rigid: i guess nmlcfgsvr was meant to do stuff like that originally. but variable processnames (i.e. emc1, emc2, ... xemc1, xemc2, ...) seem like the cleanest way to me.
06:25 PM rmu: rigid: every instance gets their own shared mem segments and tcp port
06:27 PM rmu: even with the shared memory that is used by nml/cms/rcs stuff is copied around. emc status has to be requested, then you wait for answer, then you read answer from tcp or shared memory
06:28 PM rigid: are tcp ports respected with LOCAL process types?
06:28 PM rmu: that will probably not get much worse with unix domain sockets
06:28 PM rmu: no the local stuff uses SYSV semaphores and the shared memory afais
06:29 PM rigid: remote stuff also uses semaphores.
06:29 PM rmu: (I think zeromq is using unix domain sockets for IPC)
06:29 PM rmu: the semaphore is local
06:30 PM rigid: i strongly suppose both local and remote buffers use semaphores for concurrency handling. didn't check tho
06:31 PM rigid: at least I get semaphore output on client and server when sending/receiving NML msgs
06:31 PM rmu: yes may be, maybe even the remote comms uses the shared memory buffer for internal stuff, but you can't watch the semaphore from remote
06:32 PM rigid: you don't need to watch it. the buffers are synchronized by the RCS/CMS/NML layers
06:32 PM rigid: you'll get blocked until you can read/write the buffer
06:32 PM rmu: then the rcs stuff can't watch it / use it for synchronisation
06:33 PM rigid: (if it's configured readable/writable for you)
06:33 PM rmu: you can only use it locally
06:33 PM rmu: doesn't matter
06:33 PM rigid: why can't it watch it?
06:33 PM rigid: it's a vital core concept
06:33 PM rmu: what i wanted to say is that copying into and out of shared memory is not faster than putting stuff through a pipe.
06:34 PM rigid: write the buffer, issue update, all readers can read it as if it would be local shared mem
06:34 PM rigid: faster? I didn't mention speed
06:34 PM rmu: you can't talk to the semaphore on another system. you need som e sort of TCP
06:34 PM rigid: i don't see speed as issue at all
06:35 PM rmu: you didn't mention speed, but i did. there is no need for shared memory for the messaging stuff.
06:35 PM rigid: what do you mean? you don't need to "talk to the semaphore". nmlbuf does that for you on both sides
06:35 PM rmu: no need to implement shared mem in zero mq
06:35 PM rmu: that's what i meant
06:35 PM rmu: semaphores are there for synchonisation of shared memory access
06:36 PM rigid: then you need to translate zeromq messages to something similar to e.g. emcStatus (that can be handled by all applications, python bindings etc.)
06:36 PM rigid: right now you got simple buffer that can be anything (i.e. a C++ object)
06:36 PM rmu: yes, i would stuff flatbuffers into zeromq or protocolbuffers
06:36 PM rigid: exactly
06:37 PM rmu: then you could even talk to browsers with minimal effort, flatbuffers ans zeromq is already there
06:37 PM rigid: and I think it's quite hard to implement the current design.
06:37 PM rmu: nobody will implement nml in javascript
06:37 PM rigid: i.e. transferring multiple object attributes in a single message
06:37 PM rmu: why should that be hard
06:38 PM rigid: actually librcs supports HTTP APIs... i've noticed it when looking through the sources
06:38 PM rmu: it's basically translation of emc_nml.hh
06:40 PM rmu: grep for http in libnml comes up empty except 2 links in gomath.c
06:40 PM rigid: it's hard because you need to re-implement the RCS and some stuff of the CMS layer and maybe even from the NML layer. Like locking won't come for free.
06:40 PM rmu: what locking
06:40 PM rigid: rmu: rcslib != libnml
06:40 PM rigid: the semaphore stuff we talked about
06:40 PM rigid: gotta run, ttyl
06:41 PM rmu: the rcs in linuxcnc is part of libnml dir
06:41 PM rmu: yes gn8 have to go also
06:41 PM rmu: ttyl
07:59 PM linuxcnc-build2: Build [#1789](http://buildbot2.highlab.com/buildbot/#builders/14/builds/1789) of `10-rip.debian-10-buster-rtpreempt-amd64` 8completed with warnings.
08:00 PM linuxcnc-build2: Build [#1790](http://buildbot2.highlab.com/buildbot/#builders/7/builds/1790) of `10-rip.debian-11-bullseye-rtpreempt-amd64` 8completed with warnings.
08:41 PM linuxcnc-build2: Build [#1870](http://buildbot2.highlab.com/buildbot/#builders/13/builds/1870) of `00-checkin` 3completed successfully.
09:22 PM linuxcnc-build2: Build [#1791](http://buildbot2.highlab.com/buildbot/#builders/7/builds/1791) of `10-rip.debian-11-bullseye-rtpreempt-amd64` 4failed.
09:55 PM linuxcnc-build2: Build [#1790](http://buildbot2.highlab.com/buildbot/#builders/14/builds/1790) of `10-rip.debian-10-buster-rtpreempt-amd64` 4failed.
09:57 PM linuxcnc-build2: Build [#1871](http://buildbot2.highlab.com/buildbot/#builders/13/builds/1871) of `00-checkin` 4failed.
10:24 PM linuxcnc-build2: Build [#1791](http://buildbot2.highlab.com/buildbot/#builders/14/builds/1791) of `10-rip.debian-10-buster-rtpreempt-amd64` 8completed with warnings.
10:26 PM linuxcnc-build2: Build [#1792](http://buildbot2.highlab.com/buildbot/#builders/7/builds/1792) of `10-rip.debian-11-bullseye-rtpreempt-amd64` 8completed with warnings.
10:27 PM linuxcnc-build: build #3568 of 1660.rip-buster-python3 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1660.rip-buster-python3/builds/3568 blamelist: Phillip Carter <phillc54@users.noreply.github.com>
11:04 PM linuxcnc-build2: Build [#1872](http://buildbot2.highlab.com/buildbot/#builders/13/builds/1872) of `00-checkin` 3completed successfully.
11:26 PM linuxcnc-build: build #3970 of 1640.rip-buster-rtpreempt-amd64 is complete: Failure [4failed compile runtests] Build details are at http://buildbot.linuxcnc.org/buildbot/builders/1640.rip-buster-rtpreempt-amd64/builds/3970 blamelist: Phillip Carter <phillc54@users.noreply.github.com>