Use PSRAM to keep history data

[vortigont]

Moved from #28

Hi @vortigont , thanks for your response, my idea is to buffer large data to PSRAM, as SRAM is small (around 300KB), I want to buffer the reading of PZEM every second or every minute (buffering like 10 to 20 readings), then send it to the cloud, using the internal RAM, puts a pressure on ESP32 and slow everything down, is my idea feasible ?

[vortigont]

Actually espem is already working exactly like this.
By default it keeps last 300 reading (1 per sec for 5 min) in TSContainer struc allocated in PSRAM if available.
You can check
ESPEM::wsamples method how it works to go through data container to feed the data to webui.

[Alaa-Aim]

Actually espem is already working exactly like this. By default it keeps last 300 reading (1 per sec for 5 min) in TSContainer struc allocated in PSRAM if available. You can check ESPEM::wsamples method how it works to go through data container to feed the data to webui.

Many thanks for the explanation , I read ESPEM::wsamples and it's defiantly do what I wanted, but I've tried to send data directly to the cloud, and everything seems to froze, I had to change the duration to 1 sample sent every two minuses , the heap seems not effected.

[vortigont]

Not sure what type of cloud API you are using. Maybe it's not that good to work with micro-controllers. But as you see with my project it has not problem to send 300+ data points via chunked json withing a second or less. So it is definitely not a PSRAM issue.

[Alaa-Aim]

I'm sending data using HTTPClient library to google sheets , I'll try to check everything else, many thanks for all your efforts and hard work.

[vortigont]

HTTPClient is not best choice for such tasks, it could block the MCU for a long time.
Could be better to use some intermediate local executed script that pulls all datapoints from mcu via json, than converts and uploads it to the cloud.

[Alaa-Aim]

I'll try that, thanks for the advise

HTTPClient http;
http.begin("http://espem/samples.json");
int httpCode = http.GET();
if (httpCode > 0) {
String payload = http.getString();
StaticJsonDocument<256> doc;
DeserializationError error = deserializeJson(doc, payload);
if (error) {
Serial.println("Error parsing JSON");
return;}
int Voltage = doc["U"];
int Current = doc["I"];
int Power = doc["P"];
int Energy = doc["W"];
int Frequency = doc["hz"];
int PowerFactor = doc["pf"];
HTTPClient cloud_http;
cloud_http.begin("i.e. https://google.com");
cloud_http.addHeader("Content-Type", "application/json");
cloud_http.addHeader("Authorization", "Bearer " + String(cloud_api_key));
StaticJsonDocument<256> data;
data["U"] = Voltage;
data["I"] = Current;
data["P"] = Power;
data["W"] = Energy;
data["hz"] = Frequency;
data["pf"] = PowerFactor;
String json_str;
serializeJson(data, json_str);
cloud_http.POST(json_str);
http.end();
delay(1000);}

such code could work? please advise.

[vortigont]

No, it won't work. You try to download huge json object in RAM, while ESPEM::wsamples method works by generating json on-the-fly using only a small block of RAM for a limited number of samples at a time.

[vortigont]

I mean you could create and run some other script on python or any other lang and run it from outside the ESP32 device to do the hard work talking with cloud.

[Alaa-Aim]

@vortigont many thanks. I'll try to use another ESP32 to send the data to the cloud.