#linuxcnc-devel Logs

Oct 14 2023

03:28 AM -!- #linuxcnc-devel mode set to +v by ChanServ
04:39 AM Tom_itx is now known as Tom_L
04:42 AM rene-dev5: Feel free to open an issue about it and assign it to me
05:20 AM linuxcnc-build2: Build [#1698](http://buildbot2.highlab.com/buildbot/#builders/7/builds/1698) of `10-rip.debian-11-bullseye-rtpreempt-amd64` 4failed.
05:26 AM linuxcnc-build2: Build [#1704](http://buildbot2.highlab.com/buildbot/#builders/11/builds/1704) of `10-rip.debian-12-bookworm-rtpreempt-amd64` 8completed with warnings.
10:15 AM rigid_: hm, is linuxcncsvr actually used for something? launching it seems useless currently.
10:30 AM Tom_L: i've never heard of anyone using it
10:38 AM rigid_: it's acting on behalf of remote processes. It only needs to run when there's a "REMOTE" process in the nml configuration.
10:38 AM rigid_: If I see that right, it's mostly be broken since around 2.7
10:39 AM rigid_: the actual design is pretty nifty
10:51 AM rene-dev5: Oh, is that another thing that can just go away?
11:46 AM seb_kuzminsky: i think it sets up shm used by nml
11:46 AM seb_kuzminsky: but i may be misremembering
01:27 PM rigid_: nope, that happens outside of linuxcncsvr. at least it's meant to. this is the actual top-level design: https://web.archive.org/web/20000919042916im_/http://www.isd.mel.nist.gov/projects/rcs_lib/fig1.gif
01:29 PM rigid_: rene-dev5: if you re-impliment all the locking, keying and queueing. if NML is finally removed and all bugs caused by this are fixed, it should end up as a poor libnml version without socket support.
01:30 PM rigid_: just that it's manually written instead of that java code generator that was used back then
01:32 PM rigid_: ...actually pretty smart to autogenerate it, for the time. Not surprising it was originally invented by some u.s. air force program.
02:06 PM rmu: rigid_: replacement, if it ever happens, would be something like protobuf for serialization and zeromq for transport
02:08 PM rmu: and that picture is no top level design of linuxcncsvr nor emc / linuxcnc
02:10 PM rigid_: rmu: i thought about rabbitmq at first but it's not realtime capable. After reading this [1] I have a hard time thinking of any capable replacement at all. [1] https://www.researchgate.net/profile/Jl-Michaloski/publication/243775807_The_Neutral_Message_Language_A_Model_and_Method_for_Message_Passing_in_Heterogeneous_Environments/links/00b7d533d578268915000000/The-Neutral-Message-Language-A-Model-and-Method-for-Message-Passing-in-
02:10 PM rigid_: Heterogeneous-Environments.pdf
02:10 PM rmu: rigid_: realtime doesn't matter over network
02:11 PM rigid_: rmu: sure it is. you can see it in the code.
02:11 PM rigid_: rmu: realtime doesn't matter over network?
02:11 PM rigid_: what do you mean?
02:11 PM rmu: ethernet is no realtime capable system
02:11 PM rmu: nor is tp
02:11 PM rmu: tcp
02:12 PM rigid_: wtf are you talking about?
02:12 PM rmu: not if you include switch/router/other computers on same collision domain
02:13 PM rigid_: ethernet: 1. TCP is not bound to ethernet, 2. there is ethernet over PCI for example. more than realtime capable
02:13 PM rigid_: TCP: 1. it is realtime capable (lookup RTP) 2. linuxcnc is not limited to TCP
02:15 PM rmu: RTP is UDP, and doesn't hurt anybody if packets are dropped or arrive out of order
02:16 PM rmu: ethernet can work in limited circumstances, like direct connection to mesa cards or with ethercat or with special switches, but in the general case, IP over ethernet and esp. TCP is not hard realtime capable
02:16 PM rigid_: sry, meant to say RTSP
02:16 PM rmu: nobody wants dropped packets to ESTOP their machine
02:17 PM rigid_: yeah, TCP is nice to avoid/detect dropped packets
02:18 PM rigid_: and ensuring sequence (other than plain shm)
02:18 PM rmu: the consumer of a TCP stream doesn't get notified if packets are dropped, the connection just hangs until everything is retransmitted
02:18 PM rigid_: i mean... linuxcnc has TCP via localhost actually configured by default.
02:18 PM rmu: no realtime stuff happens over that network
02:18 PM rmu: connection
02:20 PM rigid_: currently nothing happens over that network connection. but there once was realtime stuff happening over that network.
02:21 PM rigid_: and i wonder how UDP consumers get notified about dropped packets
02:22 PM rmu: they don't, UDP packets are dropped, that's it
02:22 PM rigid_: anyway. the paper says "The Neutral Message Language (NML) is a uniform API to communication functions that includes many popular protocols: interprocess shared memory; backplane global memory; and UDP and TCP/IP networking." and there's timestamping code so I guess TCP is perfectly fine for realtime at the scale that linuxcnc needs.
02:26 PM rmu: it's an unmaintained >20 year old serialization library
02:28 PM rmu: on-the-wire structure is neither self-describing, doesn't have forward or backward compatible versioning included, and AFAIK only java and C(++) interfaces exist
02:30 PM rmu: rigid_: did you have a look at that https://machinekoder.com/machinetalk-explained-part-1-introduction/
02:46 PM rigid_: rmu: we use a re-implementation of the library "maintained" by linuxcnc. The NML protocol is feature complete. Age is not really a metric I would take into account. How would self-describing messages help in the case of linuxcnc? Versioning seems to already be implemented in linuxcnc. Sure it would be nice to outsource all that, but I don't see any alternative.
02:47 PM rigid_: I looked over machinekit and found it's less tried & proven than the libnml part in linuxcnc and pretty much unmaintained. If it was maintened, I would consider it.
02:48 PM rmu: you would want a remote GUI to not just crash if something changes on the machine (or worse, crash the machine)
02:48 PM rigid_: tbh, the choice between cleaning up libnml and using machinetalk in linuxcnc is a close one
02:51 PM rigid_: i wouldn't want a remote GUI with incompatible protocol version to even work. Sure, forward compatibility is nice but pretty unnecessary in a high-availability m2m environment.
02:52 PM rigid_: i mean, it's possible with current libnml i guess. but until now, nobody ever needed this
02:56 PM rmu: don't see how that would be possible with nml, if you change something in your update() methods the wire format changes and you don't know it
03:02 PM rigid_: i think that's what CMS->min_compatible_version is meant for. it's currently just checked at connect.
03:03 PM rigid_: https://github.com/LinuxCNC/linuxcnc/blob/master/src/libnml/cms/cms.hh#L444
03:17 PM rmu: to me that looks like the version of CMS
04:35 PM rigid_: yes. For each CMS instance.
04:37 PM rigid_: It can be processed during connect/negotiation.
04:37 PM rigid_: no one ever needed a max_compatible_version property. would be trivial to add.
04:40 PM rmu: what you want is a version per message and a way to deal with new fields
04:51 PM rigid_: why per message in a stateful environment??
04:51 PM rigid_: and I doubt anyone would want that. It would be there already if otherwise.
04:52 PM rigid_: seems more like covert buzzword bingo
04:56 PM rmu: of course it is not necessary. but if you really want to do "remote" stuff you might want to talk to machines with different linuxcnc versions with the same software
04:58 PM rigid_: yeah, but not with incompatible protocols and certainly not with some complex handling just to save one update. the realtime "remote stuff" already worked fine at some state without all that. I just want it to work again.
04:59 PM rigid_: while fancy message versioning might not hurt, it's a good thing to save complexity. I'd rather strip down libnml than extend it.
05:04 PM rmu: i'm not sure what you are talking about, but AFAIK the realtime stuff uses shared memory and thats like that since before the rename to linuxcnc. up until recently there even was no real way to use the network from the realtime parts.
05:16 PM rmu: basically everything in c++ is outside the realtime domain
05:39 PM rigid_: everything uses shm (unless configured otherwise in the nml config). You could use the network part for all the buffers before. Do you mean linuxcnc, emc2 and emc never used the NML API although libnml was included?
05:39 PM rmu: not for realtime comms
05:39 PM rigid_: all of the original stuff must have worked.
05:40 PM rigid_: why not? rtapi_app is a new thing
05:40 PM rigid_: and NML was specifically chosen for this
05:40 PM rmu: realtime code was kernel code and there you can't easily use c++
05:41 PM rigid_: we've been there already. that's not true. Both RTAI and rt-preempt support userspace rt code
05:41 PM rmu: yes, now
05:41 PM rmu: that is "recently"
05:41 PM rmu: not sure about RTAI
05:41 PM rmu: back in the day, all realtime stuff was kernel modules
05:42 PM rmu: no c++ there, no network, no nml