#linuxcnc-devel Logs

May 26 2017

08:00 AM jepler: .. increasing servers from 8 to 12
08:26 AM archivist: have yo looked at the mysql slow query log to see if that is part of the problem?
08:27 AM jepler: the only mysql log is error.log and it is empty
08:28 AM archivist: try to enable the slow query log
08:29 AM archivist: the number of requests graph I saw earlier seemed reasonable though
08:31 AM jepler: # tail mysql-slow.log
08:31 AM jepler: /usr/sbin/mysqld, Version: 5.5.55-0ubuntu0.14.04.1-log ((Ubuntu)). started with:
08:31 AM jepler: Tcp port: 3306 Unix socket: /var/run/mysqld/mysqld.sock
08:31 AM jepler: Time Id Command Argument
08:31 AM jepler: # grep -A1 slow /etc/mysql/my.cnf
08:31 AM jepler: log_slow_queries = /var/log/mysql/mysql-slow.log
08:31 AM jepler: long_query_time = 2
08:34 AM archivist: 2 seconds, that is an age
08:34 AM jepler: what value do you suggest?
08:35 AM archivist: I have just rune googles pagespeed tool on a random page and it gets a score of 56/100
08:36 AM archivist: judging by the hit rate some other optimisations should be done first
08:36 AM jepler: ¯\_(ツ)_/¯
08:36 AM archivist: enable compression
08:37 AM archivist: some is of the pagespeed recommendations are stuffed due to the forum choice
08:37 AM jepler: yeah this thing is not going to be a speed demon
08:38 AM jepler: I don't really care about that
08:38 AM jepler: I care about the forum literally crashing
08:38 AM jepler: lrwxrwxrwx 1 root root 30 Aug 4 2015 mods-enabled/deflate.conf -> ../mods-available/deflate.conf
08:38 AM jepler: deflate is supposed to be enabled
08:38 AM archivist: crashing meaning running out of resources due to load, fix the speed reduces the load!
08:39 AM archivist: there was some messing about to actually use that mod in the virtual host as well
08:40 AM jepler: fwiw I get "content-encoding: gzip" shown in developer tools
08:40 AM jepler: while looking at a random page in the administration interface
08:40 AM archivist: its on the includes js and css etc
08:41 AM archivist: https://developers.google.com/speed/pagespeed/insights will tell you which
08:42 AM archivist: I tested the page https://forum.linuxcnc.org/forum/recent
08:44 AM archivist: no one tool gives all the answers either
08:47 AM jepler: that's curious, I agree that e.g., http://forum.linuxcnc.org/ is not being served compressed
08:47 AM archivist: jepler, changing the low query time to half a sec will show if there is any low hanging queries
08:48 AM jepler: what will I do once I have a slow query?
08:48 AM jepler: I am not going to write any lines of php or sql
08:48 AM jepler: zero lines
08:55 AM archivist: which mpm is apache using
08:55 AM jepler: the one you have to use with mod_php as far as I've read
08:56 AM archivist: people are recommending a change these days
08:56 AM jepler: anyway mod_prefork
08:59 AM archivist: it is something like worker and change to php-fpm
08:59 AM archivist: http://drupalwxt.github.io/performance/apache-fpm/
09:00 AM archivist: disclaimer, seen it recommended never done it
09:01 AM archivist: reading that page, how many cores are we on?
09:01 AM jepler: 1
09:02 AM archivist: dont bother then
09:02 AM seb_kuzminsky: jepler: i occasionally get >20 seconds page load times (front page)
09:03 AM archivist: when in swap probably
09:04 AM jepler: it looks like current advice is to not enable compression for https URLs (ssl-layer or http-layer), due to CRIME and BREACH attacks
09:05 AM archivist: https is there to slow everything to a crawl, do we need or want it?
09:05 AM jepler: anyway what I'm seeing right now is most compressible things compressed via http URLs and only HTML (or maybe PHP output?) compressed over https
09:06 AM jepler: sure we do, people transmit their passwords to the website
09:06 AM jepler: we don't force people to https
09:06 AM jepler: (except admins using the administrative interface)
09:07 AM archivist: for such a simple page the byte count is pretty large https://www.webpagetest.org/result/170526_E0_30JR/
09:08 AM jepler: *shrug*
09:08 AM jepler: seems unlikely to be fixable while writing 0 lines of php
09:09 AM jepler: afk
09:09 AM archivist: its the CSS
09:10 AM archivist: 570k !
09:10 AM jepler: OK same difference
09:10 AM jepler: that shit sandwich comes direct from our masters
09:10 AM archivist: they need a kick up the rear end
09:11 AM jepler: perhaps you should go let them know
09:11 AM jepler: *sigh*
09:11 AM jepler: sorry, I shouldn't be a jerk to you just because of this forum software I'm saddled with
09:12 AM jepler: afk
09:13 AM archivist: I find it hard to help when people choose all in one crap
09:14 AM archivist: bootstrap is some junk the forum writers have used to lazily get a fluid design for mobile
09:47 AM jepler: The software choice wasn't mine. I inherited the forum, which had already been running for years on joomla+kunena, and it was an undesirable outcome to lose all the existing posts and user accounts.
10:22 AM seb_kuzminsky: jepler: our users sure seem to like it, so thanks for doing that thankless job
10:30 AM archivist: what I cannot fathom is the need for half a megabyte of css for a mere few k of data
10:59 AM seb_kuzminsky: jepler, archivist: i turned on the joomla debug console for forum super users, it's at the bottom of the forum pages
11:00 AM seb_kuzminsky: not sure if it's useful
11:00 AM seb_kuzminsky: i followed these instructions: https://extensions.joomla.org/blog/item/debug-system/
11:00 AM seb_kuzminsky: i found the suggestion to turn it on here: http://www.kunena.org/docs/en/troubleshooting/improving-performance
11:04 AM seb_kuzminsky: that cool web page analyzer that archivist linked to shows that flo doesn't use client-side caching at all: https://www.webpagetest.org/performance_optimization.php?test=170526_E0_30JR&run=1#cache_static_content
11:05 AM seb_kuzminsky: per the "Cache static files in browser" section of the kunena "improving-performance" page, i want to try enabling mod_expires
11:05 AM archivist: that and googles pagespeed are helping me optimise a company brochure site at the moment
11:05 AM jepler: seb_kuzminsky: be my guest
11:05 AM seb_kuzminsky: jepler: i made the link to enable it, do i need to do anything special to restart apache? or /etc/init.d/apache reload?
11:06 AM archivist: sudo ....
11:06 AM jepler: seb_kuzminsky: apache2ctl graceful
11:06 AM seb_kuzminsky: thx
11:06 AM jepler: seb_kuzminsky: and you should have enabled with a2enmod
11:07 AM archivist: also may need to fiddle with the virtual host as you can vary the caching per type
11:16 AM archivist: pagespeed has improved 56-62 out of 100 for desktop, but you dont want to know the mobile score
11:19 AM seb_kuzminsky: archivist: i enabled caching in .htaccess, per the instructions here: http://www.kunena.org/docs/en/troubleshooting/improving-performance
11:19 AM seb_kuzminsky: what do i need to do to the virtualhost?
11:19 AM archivist: one should use the main configs not htaccess for speed
11:20 AM archivist: htacces has to be parsed every request
11:21 AM archivist: so whatever you did in htaccess do in the virtualhost
11:21 AM seb_kuzminsky: do the ExpiresByType lines just go in the VirtualHost config?
11:21 AM seb_kuzminsky: ah, ok
11:21 AM seb_kuzminsky: thanks!
11:21 AM archivist: then throw htaccess as far as you can
11:22 AM archivist: people use it because often they have no access to the main config
11:23 AM seb_kuzminsky: we have one vhost for 80 and one for 443, i guess i should put the expires config into both?
11:24 AM archivist: probably yes
11:29 AM seb_kuzminsky: hm, i think i should put it in /etc/apache2/conf-enabled/expires.conf, and it should get picked up by both
11:29 AM * seb_kuzminsky <-- not an apache admin
11:30 AM archivist: that sounds like a debianism
11:30 AM archivist: main config applies to all sites, that file is just an include to main file
11:33 AM seb_kuzminsky: it freaks me out to edit config files that are not in git
11:33 AM seb_kuzminsky: how will i know what i did, 5 minutes from now?
11:34 AM archivist: you can copy them and keep in git
11:34 AM archivist: syntax is tested when you restart, you soon know of an error :)
11:37 AM archivist: retest still has lots of red x's
11:39 AM seb_kuzminsky: yeah, i have not succeeded in getting mod_expires configured yet
11:44 AM archivist: as the static is the bulk of a page it should make a difference to apache, the data transfer and disk load, but that still leaves the memory use per page load, do we have any unused modules to throw out?
11:44 AM jepler: # (cd /etc/apache2/mods-enabled/; ls *.conf)
11:45 AM jepler: alias.conf autoindex.conf deflate.conf dir.conf mime.conf mpm_prefork.conf negotiation.conf php5.conf reqtimeout.conf setenvif.conf ssl.conf status.conf
11:45 AM seb_kuzminsky: jepler: i put it in conf-enabled, was that wrong?
11:46 AM seb_kuzminsky: i can't even tell if mod_expires is getting loaded...
11:46 AM jepler: seb_kuzminsky: you should remove the symlinks you made manually and use a2enmod (/conf/site) as appropriate
11:46 AM seb_kuzminsky: apache2ctl -M
11:46 AM seb_kuzminsky: ok it's there
11:46 AM seb_kuzminsky: jepler: i'll do that
11:49 AM seb_kuzminsky: ok done
11:49 AM archivist: enabling is separate from configuring
11:49 AM seb_kuzminsky: yeah
11:50 AM seb_kuzminsky: i can see it's enabled, but i can't see if it's configured
11:50 AM jepler: anyway I set the slow time to .2s and there aren't many. Maybe 4/hour
11:51 AM jepler: (a total of 7, the first one logged about 100 minutes ago)
11:51 AM jepler: # Query_time: 0.251425 Lock_time: 0.000140 Rows_sent: 1 Rows_examined: 110803
11:51 AM jepler: full text searches of forum posts are slow of course
11:51 AM jepler: ... AND (t.message LIKE '%LASER%' OR m.subject LIKE '%LASER%') ...
11:54 AM archivist: ouch terrible search "engine"
11:55 AM archivist: that is a table scan, fulltext would be fast
11:56 AM jepler: can I fix that while writing zero lines of php?
11:56 AM archivist: like leading % forces a tablescan
11:56 AM archivist: no
11:56 AM jepler: OK, then let's move on
11:57 AM archivist: that sort of mysql usage should be banned on forums it can kill sites
11:58 AM seb_kuzminsky: i enabled mod_info, put it on /server-info, and now i can see that mod_expires has the expected configuration variables set
11:59 AM seb_kuzminsky: oh, and i can see that ExpiresActive is not set, so it gets the default value of Off
12:00 PM seb_kuzminsky: an nd now it's on, retesting...
12:01 PM seb_kuzminsky: it's better! 51/100 cached, according to webpagetest.org
12:01 PM seb_kuzminsky: bbl
12:04 PM archivist: just need to config the compression now
12:09 PM jepler: compression is configured for http connections, and verified to be working via curl. it's off for https connections, which seems to be for security reasons.
12:10 PM jepler: .. I'm seeing different results from pagespeed vs curl as far as caching is concerned
12:10 PM jepler: e.g., on the latest invocation of pagespeed it claimed that http://forum.linuxcnc.org/components/com_comprofiler/plugin/templates/default/images/avatar/tnnophoto_n.png was not cached, but curl -v shows
12:10 PM jepler: < ETag: "18c-547837cad4e3e"
12:10 PM jepler: < Cache-Control: max-age=43200
12:10 PM jepler: < Expires: Sat, 27 May 2017 04:39:09 GMT
12:13 PM archivist: serving up the css uncompressed on https is.....
12:18 PM archivist: google is bringing people into https (it did to me)
12:21 PM archivist: certainly looks better https://www.webpagetest.org/result/170526_0A_5PPF/1/performance_optimization/ on the http only except maybe the cache time could be different to get rid of the warnings, minor though
12:30 PM jepler: I changed the compression settings for https
12:37 PM archivist: I wonder if there is a cheap/free cdn, I get a fail for that too
12:41 PM jepler: let's wait a week or until the forum falls over again and decide whether there's more to do
12:43 PM seb_kuzminsky: i bet it won't fall over any more
12:43 PM seb_kuzminsky: page load times may still be long though
12:44 PM seb_kuzminsky: i fetch https://forum.linuxcnc.org/forum/recent once per minute from highlab.com, with a 20 second timeout, and i get timeouts a few times a day
12:49 PM archivist: I think it has knocked a second off the time
12:50 PM archivist: looking at the graph, the html/php/mysql is now the main slowness
12:51 PM archivist: by the way, the forum now has an error message "Strict Standards: Declaration of getReCAPTCHAtab::getDisplayRegistration() should be compatible with cbTabHandler::getDisplayRegistration($tab, $user, $ui, $postdata) in /var/www/html/components/com_comprofiler/plugin/user/plug_cbrecaptcha/cb.recaptcha.php on line 0"
12:55 PM seb_kuzminsky: archivist: are you a superuser on the forum? if so it's probably part of the debug stuff i enabled
12:55 PM seb_kuzminsky: i bet you have the debug console at the bottom of the page, too
12:55 PM seb_kuzminsky: it doesn't seem very useful, i'm going to turn it back off
12:56 PM seb_kuzminsky: i turned on etags, but google pagespeed and webpagetest.org both still complain about our short cache expirations
12:59 PM seb_kuzminsky: i turned off joomla debug and i no longer see that error at the top of the forum pages
12:59 PM seb_kuzminsky: archivist: is it gone for you too?
01:00 PM archivist: yes
01:00 PM archivist: I dunno what the best expire times are
01:02 PM seb_kuzminsky: me neither... i'm always wary of caching, it can lead to hard-to-find bugs...
01:02 PM seb_kuzminsky: i think i'm done messing with flo for now, i'm going to let it run for a couple of days and see how it goes
07:29 PM andypugh: Do the other communication protocols (EtherCAT, Profibus, etc etc) do the device discovery thing that SmartSerial does?
07:30 PM andypugh: Noting that STMBL currently uses SmartSerial as the only supported interface, it sees to be spreading outside Mesa. Yet another ISO standard in the making?
07:32 PM pcw_mesa: I think most (Ethercat Powerlinketc ) could since the higher level protocol is CAN
07:33 PM pcw_mesa: (which has a "dictionary")
09:07 PM jepler: "
09:07 PM jepler: Get ready, Debian Stretch will release on June 17!
09:07 PM jepler: "
09:10 PM Tom_L: and i just got 8 installed...
09:12 PM seb_kuzminsky: and we dont even have an rtai kernel for 8 yet