Parsing data

Upon receipt of a HTTP POST to the resource, or a periodic poll, each of the resource's parser are applied to the whole data payload. The parsed data is then added to the data stream given by stream_id.

The parsing system provide two methods to interpret values from a resource's pull/push data payload. First and most straight-forward is JSON parsing, where a parser JSON primitive has its complete path stated in a filesystem path style. For example, the floating point primitive called "temperature" situation at the top level of a JSON string would be referenced as "/temperature". If the primitive was contained in a JSON associative array called "readings" it would be called "/readings/temperature". Through this method, many different values can be parsed from a single JSON payload and turned into a stream in SicsthSense. This can be set in the parser representation through setting the primitive's path in the parser's input_parser field and setting the input_type to "application/json".

The other, more powerful, technique uses Regexs pattern matching to enable parsing of arbitrary text payloads. The input_parser need only be set the regex that you would like to attempt to match against the payload. The input_format should be set as "XXX". Both a value and a timestamp can be capture by the given regex through the use of groups. By default, the first group is assumed to be the data value and the second group to be the timestamp. This behaviour can be changed by setting the data_group and time_group fields. If the time_group does not appear, then the current time will be used by the system.

These parsing methods provide both and easy and a powerful method to parse data entering the SicsthSense system.