#garfield Logs
Nov 25 2018
#garfield Calendar
08:43 PM rue_mohr: Tom_L, do the lines end with a comma or is that left out?
08:47 PM Tom_itx: ,"","",""
08:47 PM Tom_itx: some fields are blank
08:47 PM Tom_itx: that's the end of a line
08:47 PM Tom_itx: if they're empty they use ""
08:47 PM Tom_itx: i can show you one....
08:47 PM Tom_itx: "Frey","Richard","PO Box 849","Coffs Harbor NSW"," ","2450 ","316","","","","","1","01/01/2000","12/01/2006 6Mo","","",""
08:49 PM rue_mohr: tom, try this
08:49 PM Tom_itx: i gotta leave in 5-10 min
08:50 PM rue_mohr: http://ruemohr.org/~ircjunk/not_public_dont_open/tomdata.csv
08:51 PM Tom_itx: mine are surrounded by quotes
08:51 PM rue_mohr: yes it should pull in tho
08:51 PM Tom_itx: "name","adress"
08:51 PM Tom_itx: etc
08:51 PM Tom_itx: oh and it expects the fieldnames present
08:52 PM rue_mohr: it should just take the first row as
08:52 PM rue_mohr: or its tuned
08:52 PM rue_mohr: if so , give me the field names
08:52 PM Tom_itx: i added them
08:52 PM rue_mohr: I can already tell ya its a quote mismatch
08:53 PM rue_mohr: if you count the quotes in every line, a few will be out with an extra
08:53 PM rue_mohr: or its a ' sneaking in
08:53 PM rue_mohr: or mising
08:53 PM rue_mohr: hes guessing
08:53 PM Tom_itx: i checked for ,
08:53 PM rue_mohr: ssh
08:53 PM rue_mohr: no, I'm sure its a quote mismatch
08:54 PM rue_mohr: the extra quote is causing it to eat data
08:54 PM rue_mohr: na, its a secondary quote
08:54 PM Tom_itx: 0 1 10 100 1000 1001 1002 1003 records import
08:54 PM Tom_itx: etc
08:54 PM rue_mohr: do you have the csv on a linux machine?
08:54 PM Tom_itx: no
08:55 PM rue_mohr: k
08:55 PM Tom_itx: i'm using notepad ++ and PN
08:55 PM rue_mohr: will it load that file I gave you without the quotes?
08:55 PM Tom_itx: ^^ was the results
08:55 PM rue_mohr: ah
08:55 PM rue_mohr: so, waiting
08:55 PM rue_mohr: should be 8192 lines
08:55 PM rue_mohr: 0-8191
08:56 PM Tom_itx: says 8192 entries but they're not what the file is unless they're sorted
08:56 PM rue_mohr: hmm
08:57 PM Tom_itx: yeah those imported
08:57 PM rue_mohr: ask when you have time, I'd like to makea a quote,binary,rouge tester
08:57 PM Tom_itx: somewhat
08:57 PM Tom_itx: the "" after z imported a "
08:57 PM rue_mohr: the typecasting would have had a fit
08:57 PM rue_mohr: oooo
08:57 PM Tom_itx: but it was delimited by , and "
08:57 PM rue_mohr: that would poooch it
08:58 PM rue_mohr: , is the only delim there
08:58 PM Tom_itx: doing it without the " now
08:59 PM Tom_itx: still imported a single " after z
08:59 PM rue_mohr: java bug
08:59 PM rue_mohr: to makea " they use ""
08:59 PM rue_mohr: and that throws its count
08:59 PM rue_mohr: :S WIND
09:00 PM * rue_mohr hopes that tent stays down
09:00 PM Tom_itx: i use single quotes for that
09:00 PM Tom_itx: '"'
09:00 PM Tom_itx: if i want that char
09:00 PM rue_mohr: the app prolly cant handle that
09:00 PM Tom_itx: well, in the code anyway
09:00 PM Tom_itx: it's using "" for the output
09:00 PM Tom_itx: i can probably dump that when i get back
09:00 PM Tom_itx: but i gotta go get my kid
09:01 PM rue_mohr: gnight
09:01 PM Tom_itx: back after while
09:01 PM rue_mohr: my CAM processor is getting lcoser
09:02 PM Tom_L: i could use a different db but i'd have to learn it and make sure it imported csv
09:02 PM Tom_L: sqlite or such
09:02 PM Tom_L: i don't like those, i've used them before
09:03 PM rue_mohr: le tme help clean your data
09:03 PM rue_mohr: you might need to throw a copy on a linux machine
09:03 PM rue_mohr: see if I can write soemthing that can find it
09:03 PM Tom_L: just a sec
09:03 PM Tom_L: i cleaned some of it already
09:04 PM rue_mohr: na, I'm gonna chew on the CM software
09:04 PM rue_mohr: furrywolf, its doin good!
09:05 PM Tom_itx: FWRITE(Handle, '"'+ ALLTRIM(CUST->LNAME) + '","'+;
09:05 PM Tom_itx: ALLTRIM(CUST->FNAME) + '","'+ ALLTRIM(CUST->ADDRESS1) + '","'+;
09:05 PM Tom_itx: ALLTRIM(CUST->CITY) +'","'+ CUST->STATE +'","'+ CUST->ZIP + '","'+;
09:05 PM Tom_itx: CUST->AREA +'","'+ ALLTRIM(CUST->PHONE) +'","'+ ALLTRIM(CUST->CELL) +'","'+ ALLTRIM(CUST->WK_PHONE) +'","'+;
09:05 PM Tom_itx: ALLTRIM(CUST->WK_EXT) + '","' + ALLTRIM(STR(CUST->CLASS)) + '","' +;
09:05 PM Tom_itx: DTOC(CUST->EXEMPT) + '","' + ALLTRIM(Piano1_info) + '","' +;
09:05 PM Tom_itx: Piano2_info + '","'+ Piano3_info + '","' + Note_str )
09:05 PM Tom_L: have fun with that
09:05 PM rue_mohr: 8-/
09:06 PM rue_mohr: I'll work with the datafile if you dont mind, I'll write a checker
09:06 PM rue_mohr: I'll make an app and you run the file thru it on your side
09:06 PM rue_mohr: later
09:06 PM Tom_L: i checked it
09:06 PM Tom_L: not for everything but the obvious
09:06 PM Tom_L: found a few that had , i the field
09:07 PM rue_mohr: now find the ones that have ' or "
09:08 PM rue_mohr: or \
09:27 PM rue_mohr: zhanx,
09:27 PM rue_mohr: gesture
09:28 PM rue_mohr: a gesture can come from a keyboard, a mouse, or an audio card
09:28 PM rue_mohr: yes?
09:29 PM rue_mohr: a gesture can come from a keyboards, mice, or audio sources
09:29 PM rue_mohr: close
09:29 PM rue_mohr: a gesture can come from keyboards, mice, or audio sources
09:29 PM rue_mohr: a gesture can come from keyboards, mice, video sources or audio sources
09:29 PM rue_mohr: or smell sensors
09:30 PM rue_mohr: "their dead Dave"
09:37 PM Tom_L: none of them should
09:38 PM rue_mohr: "gesture" I'm working on creating a new gui system
09:38 PM rue_mohr: and gui is nolonger a correct term
09:38 PM rue_mohr: its an application event interface
09:39 PM rue_mohr: its an application to user event interface
09:39 PM rue_mohr: not limited to one mouse, or one focus application
09:40 PM rue_mohr: something that can encapsulate audio and visual interfaces
09:46 PM Tom_L: there are a few ' but i don't use that for delimiters
09:46 PM rue_mohr: its screwing up the java app tho
09:46 PM Tom_L: to indicate foot like 10'
09:46 PM rue_mohr: and Kevin O'Kief
09:47 PM Tom_L: work around?
09:47 PM rue_mohr: pesimistic pass, remove the extra ' and see if it all imports ok
09:47 PM rue_mohr: if so, find the escape character to apply
09:48 PM Tom_L: that would suck
09:48 PM Tom_L: how does java import esc?
09:48 PM Tom_L: what format
09:49 PM rue_mohr: well, I think the default action is to allocate 4G of ram and segfault
09:49 PM Tom_L: asc 39 is '
09:49 PM rue_mohr: but they probably overloaded that
09:49 PM rue_mohr: no, just pull them out and see if it works
09:50 PM rue_mohr: if so, a random guess is that '' is '
09:50 PM Tom_L: but i need a fix for that because i need those
09:50 PM rue_mohr: find out if it fixes it first
09:50 PM Tom_L: makes sense due to your imported file
09:50 PM Tom_L: but would that keep records from not showing up at all?
09:50 PM rue_mohr: its eating them
09:51 PM rue_mohr: between the quotes
09:51 PM rue_mohr: its prolly packed a load of records into the value of a cell
09:51 PM rue_mohr: and then evaluated it to an integer
09:51 PM Tom_L: you think memento is java?
09:51 PM Tom_L: it likely uses sqlite
09:51 PM rue_mohr: typically all phone apps are
09:52 PM rue_mohr: they use java cause they can run each app in its own jvm
09:52 PM rue_mohr: keep everyone in their own box
09:54 PM Tom_L: yeah it's java
09:54 PM rue_mohr: arg, flipping _3_ functions to layout subobjects
09:54 PM Tom_L: what chars will it burp on then?
09:54 PM Tom_L: " ' \ what else
09:55 PM rue_mohr: HAHA! I GOT IT!!!!
09:56 PM Tom_L: i shouldn't start this tonight
09:56 PM rue_mohr: duh :)
09:56 PM rue_mohr: my codolences if you needed it sooner
09:57 PM Tom_L: no
09:57 PM Tom_L: but it's an annoyance
09:57 PM Tom_L: you think doubling them would fix it?
09:57 PM Tom_L: "" '' etc
09:58 PM Tom_L: i never screwed with java
09:58 PM rue_mohr: remove then to confirm its the problem
09:59 PM Tom_L: i will
09:59 PM Tom_L: i can check the o' pretty easy
09:59 PM Tom_L: filter the last name and see if any are there
10:00 PM rue_mohr: your on a road to a HEAVY morning
10:00 PM rue_mohr: I feel obliged to warn ya
10:00 PM Tom_L: yep the O' s are there
10:01 PM Tom_L: O'Neil etc
10:01 PM Tom_L: it could be the foot inch thing since those are in double quotes too
10:02 PM Tom_L: 10' 1"
10:02 PM Tom_L: nothing in the affected records looks odd though
10:02 PM Tom_L: it's gotta be something ahead of them
10:03 PM rue_mohr: the data will start dissapearing after the troubel character
10:03 PM Tom_L: i know
10:03 PM rue_mohr: so find a record thats missing and workback
10:04 PM Tom_L: i tried that already
10:04 PM rue_mohr: k
10:05 PM rue_mohr: what is the last record before the first data starts going missing?
10:05 PM Tom_L: i don't know what the first one missing is
10:05 PM Tom_L: i know some of the ones that are missing
10:05 PM Tom_L: that's what i'm working with
10:05 PM rue_mohr: do you get a whole lot of empty fields?
10:05 PM Tom_L: no, they're just not there
10:05 PM rue_mohr: hu
10:06 PM rue_mohr: right now, I can only stand with you on this
10:06 PM Tom_L: i've fixed corrupt files befroe
10:06 PM rue_mohr: later, I can write a checker you can run on linux
10:06 PM rue_mohr: thats kinda all I can do
10:06 PM Tom_L: just knowing it's java helps i think
10:06 PM Tom_L: but i'm not sure what it's burping on
10:06 PM rue_mohr: is there is a popular csv library in java, their prolly using it
10:07 PM Tom_itx: http://wiki.mementodatabase.com/index.php/Data_Sources
10:09 PM Tom_L: this one is better than the last one i used
10:09 PM Tom_L: they have cloud storage but i'm not interested in using that at all
10:10 PM Tom_L: it may take care of some of the formatting