REST Client Library for Spark

@akrusen glad you like it. All HTTP responses end with a new line AKA \n character. So the code looks for the end of the header section (two new lines in a row) and then reads until it encounters the next new line.

For the REST API cases I’ve needed to access, the new line character has worked fine. Most ones I’ve come across respond with JSON, which typically doesn’t have new lines characters.

Fully detecting the end of an http response is a lot more work than this library handles, especially if you get into chunked responses.

To get better, we could parse the content length out of the header and then use that to determine the end – that doesn’t work for chunks. Alternatively, you can read until the connection is closed, as @nmattisson looks to be doing in his library discussed here: https://community.spark.io/t/spark-core-http-client-library/3251

I will probably head toward the connection closed approach if I have a use case.

1 Like