Core stops working after a while

Hi Guys…

my core stops running after a while (may be 5 minuts to max 45 minutes)

what I’m doing is reading from several analog / digital inputs, putting them in a JSONlike string and post it once a minute to my server via TCPIPClient

I use the RGBled as some kind of visual heartbeat, Sometimes I noticed a whatchdog reboot, after this the code has continued…but sometimes the code will stop and the core looks like an idle one??

Another point I was wondering about was that I have to use unsigned int pin1=A1 if int i got an ugly Character when concatenating to my logString e.G. X1234 instead of 1234 the hex value of the ugly thing is 14??

Is there a method to trigger the watchdog api??

Is there a chance to finding out what’s going on? Any Idea welcome!

KR

Pitt

code follows:

int di02=D2;
int di03=D3;
//D0: SDA (Serial Data Line)
//D1: SCL (Serial Clock)
int do04=D4;
int do05=D5;

int led=D6;
int errled=D7;
int ai00 = A0;
int ai01 = A1;
// A3 - A5 may be used by SPI
int ai06 = A6;
const int ai07 = A7;

String logString;
unsigned int tmp=0; //??
unsigned int prs=0;
unsigned int alt=0;
unsigned int dis=0;
unsigned int btn=0;
unsigned int bln=0; // ??


unsigned long lastConnectionTime = 0;          // last time you connected to the server, in milliseconds
bool lastConnected = false;                 // state of the connection last time through the main loop
const unsigned long postingInterval =  1 *60*1000;  // delay between updates, in milliseconds

TCPClient client;


const IPAddress server(192,168,2,111); // address for couchdb server (NOT using DNS)
const char key[]= "xxxxxxxxx";

String getData() {

 
  tmp=analogRead(ai00);// "12.5";
  prs=analogRead(ai01);
  dis=analogRead(ai06);
  alt=analogRead(ai07);
  

  btn=digitalRead(di02);
  bln=digitalRead(di03);
  
  logString = "{\"type\":";
  logString += "\"logRecord\",";
  
  logString += "\"pressure\":";
  logString += prs;
  logString +=",";
  
  logString += "\"temperature\":";
  logString += tmp;
  logString +=",";

  logString += "\"altitude\":";
  logString += alt;
  logString +=",";
  
  logString += "\"digitalBtn\":";

  logString += btn;
  logString +=",";

  logString += "\"digitalBln\":";
  logString += bln;
  logString +=",";

  logString += "\"distance\":";
  logString += dis;
  
  

  logString += "}";
  Serial.println(logString);
  return logString;
}


void setup() {
    Serial.begin(19200);
    
    pinMode(led,OUTPUT);
    pinMode(errled,OUTPUT);
    
    pinMode(di02,INPUT);
    pinMode(di03,INPUT);
    pinMode(do04,OUTPUT);
    pinMode(do05,OUTPUT);
  
    
    digitalWrite(errled,HIGH);
    delay(200);
    digitalWrite(errled,LOW);
    delay(200);
    digitalWrite(errled,HIGH);
    delay(200);
    digitalWrite(errled,LOW);
    Serial.println("core ready...");
    sendData(getData());
}

void loop() {
  if(millis() - lastConnectionTime > postingInterval) {
    
    sendData(getData());
  }
  
    RGB.control(true);

    // red, green, blue, 0-255
    RGB.color(0, 20, 0);

    // wait one second
    delay(1000);

    // resume normal operation
    RGB.control(false);
}

void sendData(String logString ){

    lastConnectionTime = millis();
  int lng = logString.length() +1;
  char logRecord[lng]; 
  logString.toCharArray(logRecord, lng);
 
  if (client.connect(server, 5984)) {
    digitalWrite(led,HIGH);
    Serial.println("connecetd...");


    client.print("POST ");
    client.print("/emon/ "); // couchdatabase
    client.println("HTTP/1.1");
    client.print("Authorization: ");
    client.println(key);
    client.println("Content-Type: application/json");
    client.println("Host: home.dewaard.de"); // ??
    client.print("Content-Length: ");
    client.println(strlen(logRecord));
    client.println();
    client.println(logRecord);
    client.println("User-Agent: sparkyOne");
    client.println("Connection: close");
    client.println();
        // note the time that the connection was made:
  
    delay(1000);
    digitalWrite(led,LOW);
    client.flush();
    client.stop();
  } 
  else {
    // you didn't get a connection to the server:
    digitalWrite(errled,HIGH);
    Serial.println("connection failed");
    //Serial1.println("disconnecting.");
    client.flush();
    client.stop();
    delay(1000);
    digitalWrite(errled,LOW);
    delay(1000);
    
  }


}

It seems to be a problem of the error handling if TCPClient fails to connect? if I shut down my targetserver the code is not continue at the else part … instead it looks the control is taken by the watchdog part of the firmware? But: this is not reliable? sometimes else part, sometimes reseting? What can I do?

thx fpr helping :smiley:

Pitt

I believe you are experiencing this https://community.spark.io/t/bug-bounty-kill-the-cyan-flash-of-death/1322 or this https://community.spark.io/t/known-issue-long-delays-or-blocking-code-kills-the-connection-to-the-cloud/950

1 Like

This might be a silly thought, but how does String allocate memory for your data? Maybe there is a memory leak in the String class? Has Spark :spark: rolled their own String class?

1 Like

Since there isn't a way to get memory usage, you might (maybe, on a good day, if the planets are aligned) be able to detect memory issues by something like this in loop():

char *foo = new char[1];
// Print the pointer value of foo to the serial console
// (sorry, I don't have the time to look up the
// conversion functions atm).
delete foo;

If the pointer address keeps on increasing, there might be a leak.

This does, of course, assume that the memory allocation functions/memory usage work similarly to those on other platforms. (This might not be true on an embedded platform like the spark).

Hi @Raldus,

I suspect something like this wouldn’t work on something like the core. It’s not running a full operating system, so I suspect some stuff (like de-fragging available ram) is not yet available. We’ve been talking about a port to a real-time operating system that would help provide some features like this. – Someone please correct me if I’m wrong.

edit: @zachary or @satishgn could provide more details on that

Thanks,
David

Well, it’s not a function of a full operating system. The memory allocation routines just have to re-use freed/deleted memory (coalescing adjacent memory blocks – “defragmenting” – is not necessary, but is highly desirable). Even in the presence of other memory allocations, that pointer value will probably bounce around, but should hopefully/eventually stabilize to remain within some range of memory.

Edit: if it doesn’t stabilize to remain within some range, then there might be a leak, either real or effective.

Well, this does assume that:

  1. Other memory allocations don’t significantly fragment memory, in the absence of adjacent memory block coalescing (“defragmenting”). If freed blocks aren’t coalesced (defragmented) – which, as you say, is certainly possible (it’s not essential and could still be on the backlog) – and memory becomes sufficiently fragmented, then this effectively acts as a memory leak. This WILL eventually kill the app.

  2. Memory is actually freed when free()/delete are used. If free/delete are a no-op (unlikely, but I haven’t looked at the spark source code), then any memory allocation in loop() – which includes all C++ objects like Strings – will eventually kill the app.

Good suggestion @Raldus!

char *foo = new char[1];
Serial.print("Pointer foo: 0x"); Serial.println((unsigned long)foo,HEX);
delete foo;

Running this in a tight loop produces:

Pointer foo: 0x20002428
Pointer foo: 0x20002428
Pointer foo: 0x20002428
Pointer foo: 0x20002428
Pointer foo: 0x20002428
Pointer foo: 0x20002428
Pointer foo: 0x20002428
Pointer foo: 0x20002428
Pointer foo: 0x20002428
Pointer foo: 0x20002428
Pointer foo: 0x20002428

Making Spark.function() requests while that is happening did not change the pointer address.

Try it in your app @pdewaard

void loop() {
  if(millis() - lastConnectionTime > postingInterval) {
    sendData(getData());
  }

  RGB.control(true);
  // red, green, blue, 0-255
  RGB.color(0, 20, 0);
  // wait one second
  delay(1000);
  // resume normal operation
  RGB.control(false);

  // If this starts spitting out larger and larger addresses... you are running out of RAM!
  char *foo = new char[1];
  Serial.print("Pointer foo: 0x"); Serial.println((unsigned long)foo,HEX);
  delete foo;
} 
1 Like

Thank you so much for testing this! Science!

1 Like

Thanks for doing the testing! :sunny:

Just for emphasis, however: in a more complex program, those pointer values might vary (they won't remain the same), but they should stay within a certain region of memory. If there's a trend upwards (or downwards on some rare platforms -- I'm not sure what the spark does), then there's likely a memory leak.

1 Like

Hi Guys, after tinkering around, i would say it’s the TCPIPClient, that fails if the connect doesn’t work in an nearly timeframe!

I included the foo mimik, the pointer is stable, I also delete now my char *logRecord[] and reinstantiate it after every http-post; looks to be stable too…

But if I shutdown the targetserver, the code doesn’t reach my else part, the core is going flashing cyan, then some red blinks, the watchdog fires, my code is starting (two blue blinks) and then he is away and looks idle, I have to press reset TWO times to get it back up and running!!

@Spark Guys, can you investigate?? or have to wait some days/weeks to go ahead with my project??

KR

Pitt

AND: Thank you all for helping! :smile:

I’m actually fairly certain that your issue relates to this thread:

The connect() call of the CC3000 blocks for up to 60 seconds. During this time period, the Core can quietly lose its connection to the Cloud.

We are working on a fix, but it’s a relatively complex one, so we don’t have a solution yet, but it’s one of the top priorities for the team right now.

1 Like

Hi Zach, OK I’m willing to wait for the fix :smile:

For the fix: my code connects every 60 seconds, the connect itself consumes 1 second, both core and targetserver resides on the same local network and are placed nearby the AP.

Two other cores are lying around, both tinkered the blue led to high, one runs OK since 2 days, one holds the blue led to high, but gets cyan flashing after some hours?

HTH

Pitt

Hi Zach, long time ago … is there a fix meanwhile? Thx…

Hi @pdewaard,

We’re close to getting an official patch from TI, and I think the plan right now is to work on distributing the patch next sprint: https://community.spark.io/t/bug-bounty-kill-the-cyan-flash-of-death/1322/463

Thanks!
David

1 Like