Onzo Delivers! - (at last)

[GrahamW]

PS I didn't correct for Summer time!

[GrahamW]

Without wishing to scare the horses, this is a summary of the academic work http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3571813/ I think this qualifies as an interesting problem 
Graham

[is0-mick]

Hi Graham
The stored data seems a bit sporadic.

Here is a spreadsheet / graph of some of the power samples I downloaded direct from the meter's flash. Just like how the upload works.
These are the POWER_REAL_FINE which appear to be spot samples of the W at a given time interval.

http://www12.zippyshare.com/v/5748968/file.html

The data time stamp is also corrected in the data to give an actual time date:

Part of the code I did:

uint ClampCurrentTimestamp = onzo.GetRegister32(netID.sensor, (byte)sensorRegs.timestamp_l);
DateTime BaseTime = DateTime.Now.AddSeconds(-ClampCurrentTimestamp);
for (ushort i = bd.start_block; i < bd.cur_block; i++ )
            {
                byte[] block = onzo.GetBlock(BlockType.POWER_REAL_FINE, i, 1);
                List<sample> mylist = b.DecodeBlock(block, BlockType.POWER_REAL_FINE);
  
                foreach (sample samp in mylist)
                {
                    DateTime timestamp = BaseTime.AddSeconds(samp.Timestamp);
                    writetext.WriteLine("Time {0} , Energy ={1}w ", timestamp.ToString(), samp.Value);
                }
            }
            writetext.Close();



Mick
EDIT:
Forgot to add, we can sample the current power / temperature as often as we want and store it on the pc, for a faster sample rate. eg at 1s resolution.

Mick

[GrahamW]

Hi Mick, Thanks for the speedy response
I don't think it's just sporadic, I think this is data compression, i.e. don't output a value that doesn't change.

So are there just three types of sample - standard, fine & reactive or are their others? Which type is the data in your spreadsheet?

I guess if the disagggregation isn't going to miss something useful, what's needed is a set of rows of data for a given period, which looks a bit like

<timestamp>, <sample type1>,<sample type2>,.....<sample type n>

I am not clear what the difference is between fine and standard, is it the time granularity? if so what are the grain sizes 'n' secs and 1 sec or something different?

Regards
   Graham

[GrahamW]

This is the data that comes out of the OWL-160
Device
 Time
 GHG_Factor
 Tariff_Cost
 Amps_Raw_Data
 Amps_Raw_Data_Min
 Amps_Raw_Data_Max
 kW_Raw_Data
 kW_Raw_Data_Min
 kW_Raw_Data_Max
 Cost_Raw_Data
 Cost_Raw_Data_Min
 Cost_Raw_Data_Max
 GHG_Raw_Data
 GHG_Raw_Data_Min
 GHG_Raw_Data_Max
Most of it can be ignored!

[is0-mick]

I don't think it's just sporadic, I think this is data compression, i.e. don't output a value that doesn't change.

Well...I thought that at first, but lots are the same value even thought the time period changes.
for instance:

 01/11/2013 00:04:33 1091
 01/11/2013 00:04:34 1091


 01/11/2013 00:08:33 1107
 01/11/2013 00:08:36 1107

 01/11/2013 00:08:37    1191
 01/11/2013 00:08:40    1191

 01/11/2013 00:08:41    1163
 01/11/2013 00:08:56    1163

So it doesn't really make sense.
The onzo seems to write to 5 data streams at the same time.

   public enum BlockType
    {
        ENERGY_LOW_RES = 1,
        ENERGY_HIGH_RES = 2,
        POWER_REAL_STANDARD = 3,
        POWER_REAL_FINE = 4,
        POWER_REACTIVE_STANDARD = 5
    }

Are the names of them.

I am not clear what the difference is between fine and standard, is it the time granularity?

Well, more we look into this, the more a mess it seems to be.

Especially when in the original code are gems like:
" MM 2009-02-13 Totally broken format from current hardware\n    quirks like data in big endian, ususable header\n\n "

Power Real Fine seems to be spot wattage readings every 1 sec.
I've not looked at standard yet, so not sure on the time period on that one.

These ones are like the total usage, and counts up from when the onzo started reading. A bit like how your electric meter does.
ENERGY_LOW_RES     every 2048 seconds data is written
ENERGY_HIGH_RES    every 512 seconds data is written

I mean why have a free running counter, then have to subtract that from the current time etc etc? which must all be done at the back end?
Lots of things don't quite make sense, I mean overall it seems quite a good system, until you start digging into it.

Mick

[GrahamW]

Hi Mick
I have had a closer look at your data (4-power_real_fine) using Excel , I still think what is going on is compression, but the pairs of values could be explained by slight coding error in the device and if the power is recorded with a finer granularity than 1 watt, this could happen if you check readings are the different by rounding to nearest integer then comparing, then rounding down the figures for transmission 5.1 & 5.9 round to 5 & 6 but round down to 5 and 5.

On a different subject, there are still some slight differences in value due to the power on or off being at different times within the second being measured, look in your data on the 8th Nov at the readings for the 6th and 7th minute past midnight. This looks like a fridge cycling "on" for 2-8 seconds at a time, the difference between the "on" bar heights is probably the transition effect when it turns on or off, when the granularity is a minute this really screws things up, but with this data it's a much smaller problem.
Graham

[GrahamW]

This has a nice diagram in it about how to use the reactive power measurements
http://homes.cs.washington.edu/~sidhant/docs/ElectriSense_Journal.pdf
Graham

[GrahamW]

Mick,
I have now had a look at your reactive power readings & I have a question, do the timestamps line up the various sets of readings? I have compared data sets 4 Power real fine and 5 power reactive standard for the first 10 minutes of the 8th of November. It doesn't make much sense, I don't understand the gaps in the reactive power readings see snippet below
Date                        Day   Hour  Min    Sec    R-Power
8 November 2013   11   8   0   2   14   121
8 November 2013   11   8   0   2   15   -349
8 November 2013   11   8   0   2   16   -349
8 November 2013   11   8   0   2   17   -281
8 November 2013   11   8   0   2   18   67
8 November 2013   11   8   0   3   36   72
8 November 2013   11   8   0   3   37   174
8 November 2013   11   8   0   3   45   184
8 November 2013   11   8   0   3   46   134
8 November 2013   11   8   0   3   47   170
8 November 2013   11   8   0   8   2   185
8 November 2013   11   8   0   8   3   159

the real power has a spike at 0:2:16 and a plateau from then until 0:3:38 then back to background levels, this plateau does correspond when most of the reactive readings are taken - so maybe it does make sense with a motor turning on for 82 second, the spike would be the peak in-rush current. in which case the background saw-tooth pattern isn't the fridge  This might make more sense. In fact taking a step back and looking at the first 6 hours of the 8th the approx. 1100 watt spike only happens twice more at 04:20:27 and 05:59:36. This is where knowing what appliances are connected aids interpretation. However it does look like plotting real and reactive power clusters could be useful.

It's useful to understand the background noise, then it can be subtracted leaving a cleaner signal for further interpretation.

Graham

[is0-mick]

I think its en4rab that sent you the reactive reading

Mick

[GrahamW]

Mick you are right, I now also have the first dumps out of my own meter, so that will keep me quiet for a while

[sandrogalli]

Apologies for revisiting an oldish thread - I found this and am very interested in the later developments. I recently purchased an Onzo and would like to log daily energy usage on a personal server.

@is0-mick, GrahamW - I was wondering if you were willing to share a few tips/scripts?

Many thanks in advance,

San

[GrahamW]

San,
the first thing you need is the dumper program from enrab4, this gets the data from the Onzo and stores it as a text file. Once you have that I can help with a java program that will interpret the text file to draw the graphs and smooth the data
Regards
   Graham

[adico]

It's good that the device measures reactive power, that helps with identifying devices, the owl was much less capable (at least using the data extracted by spreadsheet from the application).

how can you identify them ? only based on reactive power

[GrahamW]

You can't identify based only on reactive power, but that with the power measurement gives a better fingerprint, and can indicate there is a motor or similar device involved. For an effective solution you would need a DB of fingerprints and a program that could disaggregate the power usage to match the fingerprints. It would be possible to build a DB of fingerprints for your own home by turning devices on and off individually (I simplify somewhat!), a general soultion would need a lot of consumers collaborating or the manufacturers to supply fingerprints to an industry standard (if such a thing exists).