EKM Push Summary API

[Jameson]

Hello John Doe, thank you for keeping this thread rolling.

We have a LAN API to the data from the Push3, and we have a way to get the 15 minute summaries from the Push3. There is no way to pull a daily summary directly from the Push3 gateway.

=======

Here is some information about the Local LAN Data API: This is a bit experimental and not as well tested as our other APIs, so please let us know if you see any issues.

Here is the format of the call, note that it uses a gateway key not an account key, the gateway key can be found in the Meter Account UI or on the Push3 gateway web UI. The gateway-local-IP-address can also be found in the Meter Account UI:

<gateway-local-IP-address>/readMeter?key=<gateway-key>&meters=<meters>&start=<start-utc>&end=<end_utc>&count=<max-rows>

Here is an example of a call that works (if you were on the same LAN):
http://192.168.0.31/readMeter?key=NjcyN ... 4&count=10

This will return the latest 10 reads from the Push3 (once per second data) in json format, the count is adjustable up to thousands.

key (required): The gateway key (not the account key or the meter key)
meters (optional): A list of one or more meters. Default is all meters.
start (optional): The start UTC timestamp (inclusive) without milliseconds
end (optional): The end UTC timestamp (inclusive) without milliseconds
count (optional): The max number of readings in returned data. Default is 1.

Heads up that this will only return meter data in json format, you would have to write software to gather this data to interpret it, store it, or display it. You may also be able to use a json to csv file converter which can be found online.

=========

Push3 LAN Summary API: To get one 15 minute summary:
You can use this to get 15 minute summary periods that are stored on the EKM Push3 gateway SD card. You can only gather one 15 minute period at a time. You cannot use this to gather hourly, daily, weekly, or monthly data.

Here is the format of the Local API 15minute json data:

<gateway-local-IP-address>/data/summaries/meter/<meters>/<year>/<month>/<day>/<file_name>.json.gz?key=<gateway-key>

<file_name> time format is YearMonthDayHourMinuteSecond
Be sure to use the gateway key, not the account key.

For example:

http://192.168.0.32/data/summaries/mete ... Zuc05wTDRU

==

The Push3 does the work to summarize the real-time data into 15 minute chunks, the work to compress the 15 min summaries into daily summaries happens in the cloud, so that is why daily summaries are not available from the gateway.

So if you want to use the Push3 gateway LAN API, in order to get a daily summary, you would have gather the first and last summary of the day (or like you say all 96 of them if you want more data). Otherwise I would use the Summary API from the cloud to get the daily summaries: https://documents.ekmmetering.com/api-d ... ary-v2-api

[johndoe531]

Thank you for the detailed and comprehensive review of the LAN-accessible API built into the Push3 itself.

I think I'll probably just use the Summary files that get uploaded automatically on a rolling basis from my Push3 (via SFTP) to a separate server on my local LAN.

[Jameson]

Yes, you can set your Push3 to send meter data to your own server via SFTP, we can set it so that it will not send meter data to our own EKM Push servers.

With SFTP, you can setup your gateway to securely send real-tme data and 15 minute summary data to your own cloud servers. This could be your own servers at AWS, Azure, etc. You can setup your Push3 gateway to send either every single meter read (once per second in most cases) to the cloud, or you can "Set archived data recording interval to match data send interval” (which is the once per minute interval Push data is sent to our EKM Push servers by default). All you need to do is:

Log into your EKM Push gateway web UI
Check Enable SFTP
Give your Host SFTP url
Provide your Username and Password
And if you want to use a public key, you have the option to provide this as well.
Click Apply

The meter data and ioStack data will go directly to your servers. You can have EKM make a setting in the EKM Push backend, so the meter data does not go to EKM Push servers if you do not want it to. Or you can have it send to both your own SFTP server and the EKM Push server at the same time.

Screen Shot 2025-01-29 at 3.10.21 PM.png (189.43 KiB) Viewed 39672 times

[johndoe531]

Thanks for the clarifications so far. A couple of additional ones requested:

I currently have the Push3 set to send 15-minute chunks of the complete reads (not just summary files) to an SFTP server under my control.

Recently there was a connectivity issue between the Push3 and my SFTP server in the cloud, so the Push3 could not connect/send via SFTP. (The EKM meter & push3 combo continued to work fine locally, i.e. metering & storing all data on the Push3.)

Upon fixing the connectivity problem between the Push3 and my SFTP server in the cloud, it has resumed sending full Push3 reads (every single read in 15-minute chunked JSON files of about 2.5 MB size per file) as expected.

But the Push3 has not yet uploaded the backlog of 15-minute chunked JSON files that were missed during the outage. Here is a the portion of an ls -al command on my personal SFTP cloud host with the discontinuity highlighted. Now that SFTP connectivity is restored, when will the Push3 decide to upload the backlog of missing JSON files?

(note, the files end in .gz because I use a cron job to gzip them once an hour, it saves a tremendous amt of disk space.)

Code: Select all

-rw-rw-r-- 1 ekmpush3 ekmpush3   24977 Feb 22 22:31 meter_00030000XXXX_20250223061500-20250223062959.json.gz
-rw-rw-r-- 1 ekmpush3 ekmpush3   24601 Feb 22 22:47 meter_00030000XXXX_20250223063000-20250223064459.json.gz
-rw-rw-r-- 1 ekmpush3 ekmpush3   24212 Feb 22 23:01 meter_00030000XXXX_20250223064500-20250223065959.json.gz
-rw-rw-r-- 1 ekmpush3 ekmpush3   24595 Feb 22 23:16 meter_00030000XXXX_20250223070000-20250223071459.json.gz
-rw-rw-r-- 1 ekmpush3 ekmpush3   24453 Feb 22 23:32 meter_00030000XXXX_20250223071500-20250223072959.json.gz
-rw-rw-r-- 1 ekmpush3 ekmpush3   25136 Feb 22 23:46 meter_00030000XXXX_20250223073000-20250223074459.json.gz
-----> Discontinuity / missing files. When will the Push3 decide to upload these to my SFTP server?
-rw-rw-r-- 1 ekmpush3 ekmpush3   24449 Feb 23 16:09 meter_00030000XXXX_20250223234500-20250223235959.json.gz
-rw-rw-r-- 1 ekmpush3 ekmpush3   23713 Feb 23 16:16 meter_00030000XXXX_20250224000000-20250224001459.json.gz
-rw-rw-r-- 1 ekmpush3 ekmpush3   23703 Feb 23 16:32 meter_00030000XXXX_20250224001500-20250224002959.json.gz
-rw-rw-r-- 1 ekmpush3 ekmpush3   23821 Feb 23 16:46 meter_00030000XXXX_20250224003000-20250224004459.json.gz
-rw-rw-r-- 1 ekmpush3 ekmpush3   24022 Feb 23 17:01 meter_00030000XXXX_20250224004500-20250224005959.json.gz
-rw-rw-r-- 1 ekmpush3 ekmpush3 2511645 Feb 23 17:16 meter_00030000XXXX_20250224010000-20250224011459.json

[Jameson]

The EKM Push3 gateway should send the data from the time that the Push3 was offline (powered up and reading the EKM Omnimeter, but not connected to the internet). This may take some time (a few hours or so), but it should send this data to your SFTP server and fill in the missing 15 minute chunks of real-time data.

Will you let us know if it is working now, or if you are still seeing these gaps in your 15 minute chunks of real-time data on your SFTP server?