Erroneous data reading on Ominmeter III

General questions and discussions focused around the EKM Metering System. Discuss your meters, installations, and meter communications here.
phillipkopp
Posts: 10
Joined: Sun Sep 28, 2014 10:28 pm

Erroneous data reading on Ominmeter III

Post by phillipkopp »

I have a Omnimeter III hooked up to a PUSH gateway. There are extended periods of time where I am sure the meter reading is not correct (see attached screenshots). It seems like its picking up a reading (i.e. the meter is taking a reading, sending it to the PUSH and pushing it into the cloud), but the reading could not be correct, its like it almost "freezes" on a particular reading (actually it slopes gently so this frozen reading is slightly changing). Then it suddenly comes back. I am never looking at it when this happens of course so I don't know what its doing in real time.

Also, there was a couple instances where it seemed to record two readings at the same moment in time! The graph actually made like a little reverse loop. Unfortunately I did not get a screen print of that at the time. It seems to work perfectly fine most of the time, so I don't think there is anything fundamentally wrong with the setup or the hardware. But something causes it to go wacky from time to time! And the gaps are pretty substantial, so this is not good.

Thanks!
Attachments
several hours missing data
several hours missing data
2014-09-28_2238.png (13.17 KiB) Viewed 14242 times
strange "stuck" reading behavior
strange "stuck" reading behavior
2014-09-28_2218.png (19.48 KiB) Viewed 14242 times
Jameson
Posts: 860
Joined: Fri Nov 04, 2011 7:42 pm
Location: Santa Cruz, CA
Contact:

Re: Erroneous data reading on Ominmeter III

Post by Jameson »

Hi phillipkopp,

We have been doing some fiddling with the Push1 to Push2 transition. What you are seeing is that 2 out of the 3 Push1 servers were down (they are back up now). So what was happening is when the EKM Widget goes to look for data it is pulling from one of the 3 servers (sometimes it is connecting to the working server and other times it is connecting to the down server). The meter data was always in the working server. Ideally the widget or the DNS would failover to the working server.

What you are seeing in your charts is gaps in the real-time data, the meter itself is still accurate from one side of the gap to the other.

Here are the URLs that are behind the Push1 io.ekmpush.com (server13 and server14 were the servers with the issue).

curl http://server9.ekmmetering.com/10068~1.xml?MTAxMDoyMDIw
curl http://server13.ekmmetering.com/10068~1 ... AxMDoyMDIw
curl http://server14.ekmmetering.com/10068~1 ... AxMDoyMDIw

Let us know if you still are seeing any issues.
Jameson
EKM METERING
http://www.ekmmetering.com
831.425.7371
phillipkopp
Posts: 10
Joined: Sun Sep 28, 2014 10:28 pm

Re: Erroneous data reading on Ominmeter III

Post by phillipkopp »

Thanks Jameson, this is a logical explanation. I will add that I had a few issues when setting up the PUSH initially and which server and version to select (version 1 or 2). I figured that since my meter and gateware were brand new that it would be Version 2. That didn't work. I had to try every combination and eventually I found the one that worked, however this was not clear in the instructions which show screen shots of an older software REV which doesn't even have a field to select version or server!

By the way, here is a capture of the "reverse loop" double reading issue. Hopefully this is caused by the same server issue. Please advise us when you will be done with all the upgrades etc...

Thanks
Attachments
multiple reading loop issue (pinpointed as single or multiple server request failure)
multiple reading loop issue (pinpointed as single or multiple server request failure)
2014-09-29_2134.png (23.43 KiB) Viewed 14229 times
Jameson
Posts: 860
Joined: Fri Nov 04, 2011 7:42 pm
Location: Santa Cruz, CA
Contact:

Re: Erroneous data reading on Ominmeter III

Post by Jameson »

That is a funky looking graph. By the way where are you getting this graph from? (the one you just posted)
which server and version to select (version 1 or 2)
Yes, we have the Dash setup so that you can get your data from 3 different URLs and in Push1 data format or Push2 data format. This is really only here for the transition from Push1 to Push2. For now you can use "1" and io.ekmpush.com
Screenshot 2014-09-30 09.04.09.png
Screenshot 2014-09-30 09.04.09.png (69.02 KiB) Viewed 14225 times
We are now beta testing Push2 with some users, assuming all goes well, we should have it rolled out to all users within a couple of weeks.

Thanks,
Jameson
EKM METERING
http://www.ekmmetering.com
831.425.7371
phillipkopp
Posts: 10
Joined: Sun Sep 28, 2014 10:28 pm

Re: Erroneous data reading on Ominmeter III

Post by phillipkopp »

The graph is coming from the daily encompass report emailed pdf file.

In my encompass.io dashboard this is also still going on in a pretty big way. See attached. Whatever your server problem is not fixed. Maybe it has something to do with the encompass platform? In my Dash software of course I can only see one reading (Watt for example) so there is no looping going on the graph.
Attachments
2014-10-01_1204.png
2014-10-01_1204.png (27.88 KiB) Viewed 14207 times
Jameson
Posts: 860
Joined: Fri Nov 04, 2011 7:42 pm
Location: Santa Cruz, CA
Contact:

Re: Erroneous data reading on Ominmeter III

Post by Jameson »

Thanks, is this happening for you right now? Is this what the encompass graph looks like when you first load it, or what it looks like after running for 30 minutes? Does the loop go away when you re-fresh the page (this is info I need to understand better what is going on)

I am not aware of a Push1 server/data issue right now.
Jameson
EKM METERING
http://www.ekmmetering.com
831.425.7371
phillipkopp
Posts: 10
Joined: Sun Sep 28, 2014 10:28 pm

Re: Erroneous data reading on Ominmeter III

Post by phillipkopp »

This was happening at that moment in time. But I left and didn't see your message. I was messing around with it and connected via USB directly to the meter everything seems to be working perfectly there. I connected it back to the PUSH gateway now and it did something similar on the chart while the PUSH gateway was unplugged. The problem is there is no way to scroll back in time so I can't get back to 10am data. This happens just in the middle of any point in time (its always running basically). It happens, then it goes away. I will really try to capture what is going on when this first starts again.
Jameson
Posts: 860
Joined: Fri Nov 04, 2011 7:42 pm
Location: Santa Cruz, CA
Contact:

Re: Erroneous data reading on Ominmeter III

Post by Jameson »

Thanks for the additional info. I have seen loops in the encompass.io real-time graphs before, usually when something was going on with our local internet network (making it so some reads make it to one of the servers in the cluster and others get lost along the way to the cloud). This could be fixed if we only retrieved data from 1 of the servers in the cluster. I would be interested in knowing if you were to ever see oddities in encompass and the widget at the same time.

In encompass, try clicking "Edit" and choosing 1000 reads to see if you can see the issue back in time.
Jameson
EKM METERING
http://www.ekmmetering.com
831.425.7371
phillipkopp
Posts: 10
Joined: Sun Sep 28, 2014 10:28 pm

Re: Erroneous data reading on Ominmeter III

Post by phillipkopp »

Interesting update. Of course I was not looking when this happened, however

1. I tried changing the read interval to 15 minutes to see if slowing the data rate down made any difference (don't know why, just curious to see what happens). This only seems to affect DASH as the online readings are still all happening at 1 min. intervals.
2. I signed up for a wattvision account. I can see in my DASH software the push to wattvision API every 15 minutes. But wattvision also picks up every minute, meaning it must be linked to your encompass servers?
3. Surprise, wattvision, widget AND encompass all show an anomaly around the same time period.

No idea still if the source of this is at the PUSH at the server or if somehow there is a data issue on my end, but I am pretty sure that the internet connection is stable, nothing else around here has been having issues.
Attachments
widget data anaomaly
widget data anaomaly
2014-10-01_1535.png (50.49 KiB) Viewed 14197 times
wattvision data anomaly around the same period
wattvision data anomaly around the same period
2014-10-01_1525.png (77.49 KiB) Viewed 14197 times
encompass data anaomaly
encompass data anaomaly
2014-10-01_1527.png (37.21 KiB) Viewed 14197 times
Jameson
Posts: 860
Joined: Fri Nov 04, 2011 7:42 pm
Location: Santa Cruz, CA
Contact:

Re: Erroneous data reading on Ominmeter III

Post by Jameson »

Thanks,

I took a look at your data stream and what I am seeing is that your EKM Push is reading your meter 100% of the time, however it is dropping its connection to the EKM Push servers once in a while (3 times lately).

If you are interested here is how I found this:

This is on a Mac so bear with me. I will use our sample meter here for illustration purposes.
In terminal run this legacy command: curl http://io.ekmpush.com/10068~1000.xml?MTAxMDoyMDIw > datagaps.xml
This will create a file named datagaps.xml that contains the latest 1000 reads from meter 10068
Then go to: http://www.luxonsoftware.com/converter/xmltocsv
Convert your xml file into a csv file.
Then open this csv file in excel.
Plot your sequence numbers. These should be in a very straight line.

Here is what yours looks like (newest reads on the left):
Screenshot 2014-10-01 15.53.27.png
Screenshot 2014-10-01 15.53.27.png (59.81 KiB) Viewed 14195 times
This could be done with all 3 servers individually, if I really wanted to dial it in:

curl http://server9.ekmmetering.com/10068~1.xml?MTAxMDoyMDIw
curl http://server13.ekmmetering.com/10068~1 ... AxMDoyMDIw
curl http://server14.ekmmetering.com/10068~1 ... AxMDoyMDIw

Cant wait for the new Push system, where you can call either xml or csv directly.
Jameson
EKM METERING
http://www.ekmmetering.com
831.425.7371
Post Reply