getEventStream data type and processing

Hi, trying to get data from the Spark cloud into our system. We’re using the SparkJS library, so went ahead and did spark.getEventStream(false, 'mine', myFunction). However, the data that gets input to myFunction is a concatenated line, of the sort event: eventNamedata: {"data":"-72","ttl":"60","published_at":"date","coreid":"0"}. In particular, event: eventName and data:[...] have been put together, which makes processing annoying (but not impossible). Do I really just need to do a bunch of string processing, or alternatively rewrite the publish function on the Spark core? It just seems so inelegant. Thanks

To further expand on this, is there any guarantee on how much data will be given to the function? It appears that getEventStream usually only sends over one or two lines, which correspond to at most one event. But on occasion, the function gets 4+ lines, corresponding to >=2 events. Again, seems like there’s a more elegant way to do this besides a ton of string processing.

@rastapasta might have some ideas since he probably have tried? :smiley:

I’ve also been struggling with the inconsistency of getEventStream. It seems the problem only occurs when the first parameter of the function getEventStream is false. If the name of the event is hard coded, then I consistently get one line each time (thus one event every two times in the function).

I want to receive more events that just one, so hard coding the event name is not an option for my application. I think there must be a bug in the way getEventStream(false, 'mine', function) is implemented compared to getEventStream('event-name', 'mine', function).

UPDATE:

I was playing around with getEventStream again, and I got the inconsistent behavior even when using the function with the event name hard coded. It just happens rarely enough I didn’t catch it initially (like 1 in 10 times). I looked at the SparkJS code and I’ve decided the inconsistency is not from the SparkJS code, but just JavaScript (or Node.js which is what I’m using).

Here is the SparkJS code:

Spark.prototype.getEventStream = function (eventName, coreId, callback) {
  var requestObj = this.api.getEventStream(eventName, coreId, this.accessToken);

  if (callback) {
    requestObj.on('data', function(event) {
      event = event.toString().replace('\n', '');
      if ((event !== ' ')&& (typeof(callback) === 'function')) { callback(event); }
    });
  }

  return requestObj;
};

I think that requestObj.on('data', function(event) is the culprit.

1 Like

Hi All!

Thanks for posting, I opened an issue for SparkJS for getEventStream here, sorry it’s not behaving as expected!

You shouldn’t have to do string processing, that’d be silly, the library should do that for you! When I get some time I’ll fix that.

Thanks,
David

3 Likes

Great, thanks for the help!