Back
[12:03:56] <psha> mhaberler: capnp was really woth reading
[12:04:14] <mhaberler> great - is it useful yet?
[12:04:16] <psha> at least it proves me that offset pointer may be implemented
[12:04:32] <mhaberler> offset pointer… who is that
[12:04:41] <mhaberler> used in his encoding?
[12:04:46] <psha> i have not yet decided if i have to use it's format as-is or just adopt some smart ideas
[12:04:56] <psha> offset pointer...
[12:05:05] <psha> assume you have message int, string, int
[12:05:06] <mhaberler> job security :-?
[12:05:09] <mhaberler> ok
[12:05:22] <psha> how do you encode it?
[12:05:29] <mhaberler> vtable
[12:05:44] <psha> if you encode as 4-bytes int, 2-byte string len, N bytes string, 4-bytes int
[12:05:45] <mhaberler> does he have a tagger pointer map or what
[12:05:50] <mhaberler> tagged
[12:05:58] <psha> it's impossible to know where exactly third field live
[12:06:21] <psha> but if you encode just like struct n { int f0; const char * str; int f1; }
[12:06:37] <psha> then you'll get 4-byte int, offset pointer, 4-byte int, string data
[12:06:48] <psha> and you cat access any fixed field with just offset
[12:06:51] <mhaberler> basically a TLV format
[12:06:58] <psha> and variable fields (string, list) with offset pointer
[12:07:38] <mhaberler> are these structs all at the beginning or interleaved with data?
[12:08:22] <psha> as i understand simple structures with scalar only fields (int, double, bool) are offset only
[12:08:25] <psha> just like plain C struct
[12:08:43] <psha> everything else - list, string, sub-structs are implemented with offset pointers
[12:09:12] <psha> again - just like in C - struct s { int i; int *int_list; struct sub * sub_field; ... }
[12:09:46] <psha> i've used such approach for presense maps on complex structures and now i see that it's possible to use for data too
[12:10:33] <psha> heh, on friedday i had discussion with my collegue about offset packed strings :]
[12:10:47] <psha> so thanks a lot for that link :)
[12:11:07] <mhaberler> the capnp infrastructure isnt there, but the protobuf is - why dont you use the capnp de/serializers with protobuf like transenc does?
[12:12:30] <psha> capnp has complex algorithm of offset calculation
[12:12:44] <mhaberler> ah, not a drop in
[12:12:49] <psha> so it's not possible to just 'encode this field, encode that fields'
[12:12:58] <psha> for pb it's ok to just append several fields one by one
[12:13:07] <psha> they are tagged and order is not imporant
[12:13:30] <psha> *) important for performance - deserialization works better when fields are encoded in order of their ID's
[12:14:08] <psha> with capnp you can not just encode them one by one - fields may be placed in padding space of earlier ones
[12:14:12] <mhaberler> does that really count, I mean a few hundred nsec up or down
[12:14:18] <psha> however - not impossible