Eco Eye PV Smart

[ianh64]

I've written a python script that I am currently testing on my new Pi. Will be happy to post on here in a few days time when its had a few more days running. Its currently just updating averaged house and gen power to pvoutput, but I am thinking of adding temp from local weather station. Having to wait for 5 minute status intervals and some brighter sky's is a bit of a pita when checking output.

@Peter_L: can I interrogate the PV Smart for things like voltage? I've hard coded 240v when converting from amps to watts but it would be nice to get this from the PV Smart itself.

-Ian

[JST200]

Hi Ian,

I would definitely be interested  in your script, if that's OK. Anything to get me started!   

I guess you know that PVOutput will auto add the weather/temperature for you from a local station, using weatherunderground as the source?

However, I too (for my VB app) coded a feed from a local station (either using google or weatherunderground as my source), adding it to my upload data. I started coding it just before bankstownbloke added it to PVOutput, so kept going for programming practice.

I don't know if this will help or is wanted or not, but pvoutput only accepts a few weather type designations so those provided by google or weatherunderground need condensing. Here's my conversion table (VB coding structure removed):

          'Google

            Value ("fine")
                becomes = "Fine"
            Value ("rain showers")
                becomes = "Showers"
            Value ("partly sunny")
                becomes = "Partly Cloudy"
            Value ("scattered thunderstorms")
                becomes = "Showers"
            Value ("showers")
                becomes = "Showers"
            Value ("scattered showers")
                becomes = "Showers"
            Value ("rain and snow")
                becomes = "Snow"
            Value ("overcast")
                becomes = "Cloudy"
            Value ("light snow")
                becomes = "Snow"
            Value ("freezing drizzle")
                becomes = "Showers"
            Value ("chance of rain")
                becomes = "Showers"
            Value ("sunny")
                becomes = "Fine"
            Value ("clear")
                becomes = "Fine"
            Value ("mostly sunny")
                becomes = "Partly Cloudy"
            Value ("partly cloudy")
                becomes = "Partly Cloudy"
            Value ("mostly cloudy")
                becomes = "Mostly Cloudy"
            Value ("chance of storm")
                becomes = "Showers"
            Value ("rain")
                becomes = "Showers"
            Value ("chance of snow")
                becomes = "Snow"
            Value ("cloudy")
                becomes = "Cloudy"
            Value ("mist")
                becomes = "Cloudy"
            Value ("storm")
                becomes = "Showers"
            Value ("thunderstorm")
                becomes = "Showers"
            Value ("chance of tstorm")
                becomes = "Showers"
            Value ("sleet")
                becomes = "Snow"
            Value ("snow")
                becomes = "Snow"
            Value ("icy")
                becomes = "Snow"
            Value ("dust")
                becomes = "Fine"
            Value ("fog")
                becomes = "Cloudy"
            Value ("smoke")
                becomes = "Fine"
            Value ("haze")
                becomes = "Fine"
            Value ("flurries")
                becomes = "Showers"
            Value ("light rain")
                becomes = "Showers"
            Value ("snow showers")
                becomes = "Snow"
            Value ("hail")
                becomes = "Snow"
            Value ("drizzle")
                becomes = "Showers"
            Value ("heavy rain")
                becomes = "Showers"

                'Wunderground

            Value ("Light Drizzle")
                becomes = "Showers"
            Value ("Light Rain")
                becomes = "Showers"
            Value ("Light Snow")
                becomes = "Snow"
            Value ("Light Snow Grains")
                becomes = "Snow"
            Value ("Light Ice Crystals")
                becomes = "Snow"
            Value ("Light Ice Pellets")
                becomes = "Snow"
            Value ("Light Hail")
                becomes = "Snow"
            Value ("Light Mist")
                becomes = "Partly Cloudy"
            Value ("Light Fog")
                becomes = "Partly Cloudy"
            Value ("Light Smoke")
                becomes = "Partly Cloudy"
            Value ("Light Volcanic Ash")
                becomes = "Partly Cloudy"
            Value ("Light Widespread Dust")
                becomes = "Partly Cloudy"
            Value ("Light Sand")
                becomes = "Partly Cloudy"
            Value ("Light Haze")
                becomes = "Partly Cloudy"
            Value ("Light Spray")
                becomes = "Partly Cloudy"
            Value ("Light Dust Whirls")
                becomes = "Partly Cloudy"
            Value ("Light Sandstorm")
                becomes = "Partly Cloudy"
            Value ("Light Low Drifting Snow")
                becomes = "Snow"
            Value ("Light Low Drifting Widespread Dust")
                becomes = "Partly Cloudy"
            Value ("Light Low Drifting Sand")
                becomes = "Partly Cloudy"
            Value ("Light Blowing Snow")
                becomes = "Snow"
            Value ("Light Blowing Widespread Dust")
                becomes = "Partly Cloudy"
            Value ("Light Blowing Sand")
                becomes = "Partly Cloudy"
            Value ("Light Rain Mist")
                becomes = "Showers"
            Value ("Light Rain Showers")
                becomes = "Showers"
            Value ("Light Snow Showers")
                becomes = "Snow"
            Value ("Light Ice Pellet Showers")
                becomes = "Snow"
            Value ("Light Hail Showers")
                becomes = "Snow"
            Value ("Light Small Hail Showers")
                becomes = "Snow"
            Value ("Light Thunderstorm")
                becomes = "Showers"
            Value ("Light Thunderstorms and Rain")
                becomes = "Showers"
            Value ("Light Thunderstorms and Snow")
                becomes = "Snow"
            Value ("Light Thunderstorms and Ice Pellets")
                becomes = "Snow"
            Value ("Light Thunderstorms with Hail")
                becomes = "Snow"
            Value ("Light Thunderstorms with Small Hail")
                becomes = "Snow"
            Value ("Light Freezing")
                becomes = "Snow"
            Value ("Light Freezing Rain")
                becomes = "Snow"
            Value ("Light Freezing Fog")
                becomes = "Snow"

            Value ("Heavy Drizzle")
                becomes = "Showers"
            Value ("Heavy Rain")
                becomes = "Showers"
            Value ("Heavy Snow")
                becomes = "Snow"
            Value ("Heavy Snow Grains")
                becomes = "Snow"
            Value ("Heavy Ice Crystals")
                becomes = "Snow"
            Value ("Heavy Ice Pellets")
                becomes = "Snow"
            Value ("Heavy Hail")
                becomes = "Snow"
            Value ("Heavy Mist")
                becomes = "Cloudy"
            Value ("Heavy Fog")
                becomes = "Cloudy"
            Value ("Heavy Smoke")
                becomes = "Cloudy"
            Value ("Heavy Volcanic Ash")
                becomes = "Cloudy"
            Value ("Heavy Widespread Dust")
                becomes = "Cloudy"
            Value ("Heavy Sand")
                becomes = "Cloudy"
            Value ("Heavy Haze")
                becomes = "Cloudy"
            Value ("Heavy Spray")
                becomes = "Cloudy"
            Value ("Heavy Dust Whirls")
                becomes = "Cloudy"
            Value ("Heavy Sandstorm")
                becomes = "Cloudy"
            Value ("Heavy Low Drifting Snow")
                becomes = "Snow"
            Value ("Heavy Low Drifting Widespread Dust")
                becomes = "Cloudy"
            Value ("Heavy Low Drifting Sand")
                becomes = "Cloudy"
            Value ("Heavy Blowing Snow")
                becomes = "Snow"
            Value ("Heavy Blowing Widespread Dust")
                becomes = "Cloudy"
            Value ("Heavy Blowing Sand")
                becomes = "Cloudy"
            Value ("Heavy Rain Mist")
                becomes = "Showers"
            Value ("Heavy Rain Showers")
                becomes = "Showers"
            Value ("Heavy Snow Showers")
                becomes = "Snow"
            Value ("Heavy Ice Pellet Showers")
                becomes = "Snow"
            Value ("Heavy Hail Showers")
                becomes = "Snow"
            Value ("Heavy Small Hail Showers")
                becomes = "Snow"
            Value ("Heavy Thunderstorm")
                becomes = "Showers"
            Value ("Heavy Thunderstorms and Rain")
                becomes = "Showers"
            Value ("Heavy Thunderstorms and Snow")
                becomes = "Snow"
            Value ("Heavy Thunderstorms and Ice Pellets")
                becomes = "Snow"
            Value ("Heavy Thunderstorms with Hail")
                becomes = "Snow"
            Value ("Heavy Thunderstorms with Small Hail")
                becomes = "Snow"
            Value ("Heavy Freezing")
                becomes = "Snow"
            Value ("Heavy Freezing Rain")
                becomes = "Snow"
            Value ("Heavy Freezing Fog")
                becomes = "Snow"

            Value ("Overcast")
                becomes = "Cloudy"
            Value ("Clear")
                becomes = "Fine"
            Value ("Partly Cloudy")
                becomes = "Partly Cloudy"
            Value ("Mostly Cloudy")
                becomes = "Mostly Cloudy"
            Value ("Scattered Clouds")
                becomes = "Partly Cloudy"

            Otherwise = "Unknown"


Cheers, Jim

[ianh64]

Im using the live status update (at 5 minute intervals). This only provides temperature input and not conditions. The weather conditions are in the end of day add output service which I have not addressed yet but assuming that cumulative calculations can be relied on, probably would not take much to implement. Having only got Pi a week ago, I've used this as a crash course into Pi and Python so am happy to post the script for others to play with, may be even later today.

Im eventually going to interface with my inverter for the generation and voltage info which will be a better place to obtain the eod bits although depending on how my calculations add up over a few days I may see about adding the add output message. I only wrote my script this weekend so your post was very pertinent, but as a result, I don't have much real data to work on and only finished the upload to pvoutput yesterday evening (after the sun had gone down) and having found a bug in solar generation this morning I have scrapped data from earlier test runs.

As I'm averaging power over a 5 minute interval, I have yet to fully see how well this tracks over longer periods of time. I may need to use another algorithm to smooth peaks and then plot instantaneous readings on the 5 minute interval. So that may need to be changed if averaging doesn't track very well.

You can see my output here

-Ian

[JST200]

Hi Ian,

Output looking good - look forward to seeing your code. I'm impressed you've only done it in a couple of days!

Something else about PVoutput that may be helpful to you. It allows you to have multiple systems defined, each identified by it's own code. So I set up a "Live" system and then a couple of dev systems,  so I can test changes against the dev systems without disrupting my Live data. Maybe not so helpful when using pi for real time data - but perhaps worth a thought?

I was wondering how you agragate the 4 sec updates into 5 min uploads.  Presumable sum the 5m of updates divided by 75 to get the average?

Cheers, Jim

[JST200]

Oh, another thing which may be obvious but caught me out... what time of day to take your snapshot of the days conditions.

If you take it at the time you do your EOD processing, then that may well be sometime in the middle of the night - which may not be representative of the day.
So I decided to take my snapshot at  15:00 - being west facing the time most likely to produce the max PV output for me. Once you get your monitoring up an running you can cross check this time with reality.

With regard to smoothing. I find that PVOutputs own smoothing algorithms do a pretty good job with short update intervals (5 - 15mins). Beyond that they get a bit steppy I think.

Cheers, Jim

[ianh64]

Im attaching my script here. Its basic but should provide equivalent functionality of that offered by Trax PVOutput upload.

Its a little in return for the advice and assistance provided by people on this forum. Even though its setup for the EcoEye PV Smart, there are a number of useful bits in the code that can still hopefully benefit those not using the EcoEye unit.

In spirit of Raspberry Pi, please feel free to experiment but please do keep to rules of GNU GPLv3 license if you decide to modify and pass on.

As far as testing after a couple of false starts, its been running most of today which is totally inadequate for production use however it seems to run reliably and produce results in the right ball park as expected. But only time will tell how well it compares to real figures. However, I have released into the wild in this state to get some early feedback.

If you get the program to auto start on Pi boot, please let me know. This would allow it to run headless with just the EcoEye USB/Serial converter plugged into the USB port and a network connector which is tbh, the final aim for me.

It requires Python3 which was on my version of Pi distributed by Maplin as the basic kit. To get it up and running, you will need to download the serial library - instruction on how to do so are in the script header. I have run it from both the IDLE3 environment and from a terminal (you need to set the script file to exec (chmod +x pvs2pvo.py) and run as ./pvs2pvo.py). Like skinning cats, there are other many alternative ways to get it up and running.

Im not convinced that the simple averaging works particularly well in the sort of weather that we have been having today - i.e. sun in/out/in/out shake it all about. I think a weighted moving average or kalman algorithm for filtering would provide a better instantaneous reading. Feel free to experiment and report back.

Currently, like Trax, it only implements the PV Output status message for intraday live readings but it would not take much to modify to add an output message for EOD. It could probably do with a bit of error handling included too, especially exception handling in the PV output connection handler.

And before you get it up and running, you will need to change a few variables at the start to suit your needs and PV Output settings.

I called mine, pvs2pvo.py when looking at the references above and below.

Script moved to Home Automation and Monitoring Section

[JST200]

That's brilliant, Ian. Thanks.

Unfortunately, being new to Pi I'm falling at the first fence

I've done the "sudo apt-get install python3-serial" which seemed to run to completion OK.

However, when I run your code in IDLE 3 I get "ImportError: no module named serial" from the "Import" line of code, which I assume means it can't find the imported serial modules.

Any pointers very gratefully received!!

Cheers, Jim

[ianh64]

Your assumption sounds correct. To resolve, try/check the following. Otherwise, its google Im afraid.

Did you definitely put the 3 on Python when you imported. Otherwise it will go to python 2.x.

Are you definitely running Python 3? On my Maplin version, it is IDLE3 icon not the IDLE icon.

Also, have you restarted Python after the install? Ie not been running python/idle in window shell whilst installing - don't know if that will make a difference.

Double check the install logs.

Also, have you updated your OS to latest version?

Of course, I may also have given wrong instructions to install serial! I've only done it once and had to search google for the instructions so I may have found wrong ones.

[JST200]

I scanned google first and the answers seemed pretty much what you have suggested.

In answer...

I'm pretty sure I did.
Definitely running Python 3 (IDLE 3) (V3.2.3 to be exact), though I did try Py2 (IDLE)with the same outcome.
Tried the install command a couple of times and pretty sure Python wasn't running at the time.
Haven't looked at the install logs - something to find and read  
Latest OS installed I think (I used the NOOBS_v1_3_3.zip) which has a release date of 24/12/2013.

Well if it worked for you it ought to have worked for me!  lol

Thanks for the help - google here we come.  


Edit/Update:  Found the  install log and guess what - you were right! Initially installed python-serial and not python3-serial. However, I did then install python3-serial but I have a feeling the first install would have mucked it all up. So next lesson, how to uninstall an install and try again!

[ianh64]

There seem to be a variety of ways of installing pyserial. The pyserial documentation only briefly hints at the method that I used. I think I found the method that I used specifically for Raspberry Pi.

I don't know if it is possible to look in the python 3 folder for the serial module.

I just searched in google for the bits missing from my knowledge, which is a lot as its 30 years since I last used unix in anger although amazing how muscle memory can still persist after all these years and the fingers still just type some relevant commands.

Give
http://www.raspberrypi.org/phpBB3/viewtopic.php?f=26&t=5938
And
http://www.raspberrypi.org/phpBB3/viewtopic.php?f=32&t=51468
a try for more info/hints.

Not sure if a mod would like to move this latest discussion to a more relevent forum - I noticed one called Home Automation and Monitoring which may be a better fit for home brew code.

[JST200]

Hi Ian,

Thanks for the pointers. I checked out those, plus quite a lot of other sites too and the general consensus seemed to be to download pyserial.zip and install it manually. Which I did.  Still no joy. Same error.   

So before I re-installed the whole system from scratch (to give me a clean slate to work from) I thought I would create a new .py file adding your code one small section at a time. I only added a library element to the import statement as it was required by the added code. After each code add I ran it to ensure it ran OK.

The only issue I found was that I needed to comment out the "com.open()" statement, otherwise I got an "serialexception: com port already open". Reading up on the error it appears the "com.serial.Serial" statement issues an open anyway (but I wouldn't swear to that!).

After adding the final "print("Processing PV Smart..." section of code, it miraculously burst into life!!!

As far as I can see there is no difference between the original code file and the built up one. Whenever I try the original code file it always displays the "serial" error - even if I completely remove the import statement. Weird!   
I knew I didn't like UNIX! Give me an OS where they use real Words for commands, rather than cryptic ones, any day!  lol

Not sure this little digression needs to be placed elsewhere - as, as far as I am concerned, the digression is finished. It's onwards and upwards now!   

With regard to running pvs2pvo.py on boot I don't know if this will point you in the right direction....

To get screen desktop access to my pi, I use the TightVNC Server on my pi and TightVNC Client on my PC. To get the server on the pi to autostart you put the file "Tightvnc.desktop" in the folder "/home/pi/.config/autostart". I think you have to manually create the "autostart" folder in the hidden folder ".config". The file contains:

[Desktop Entry]
Type=Application
Name=TightVNC
Exec=vncserver :1
StartupNotify=false

So presumably if I add something similar for pvs2pvo.py then it too will auto start. I'll give it a go later and let you know what happens - if you don't try it first. 


So thanks once again for your help!

Cheers, Jim

[ianh64]

Glad you are up and running.

It's been my first full uninterrupted day of running and I'm pleased with the tracking and inverter comparison.

My inverter reported 1.85KWh, the Smart PV display registered 1.70KWh and PVOutput reported 1.739KWh.

In addition, generation and usage tracked quite nicely as I'm running an Optimmersion that uses most of the excess generation.



I also aded an EOD update which is work in progress. It will be interesting to see what that reports for daily consumption and generation. I'm not planning on releasing that as its really a check as the intraday updates generate most of this info automatically anyhow so its kind of redundant.

I'll add an auto start script soon on my unit but I'm happy with it just sitting on my desk running merrily away. Looks like the crontab file may be simplest place for this and piping output to a log file.

I'm quite pleased with my two or three hours work and tempted to get a second Pi for other projects.

[ianh64]

Cr** - my Sky broadband is down and the useless people at Sky say it will take up to 5 working days to fix as they say there is a fault near my house. Thankfully I have temporary access via an iPhone Three Mobile teather that oddly enough, works very well even though its via a signal booster that, wait for it, uses my Sky broadband connection.

Me thinks they are either very clever at Sky having invented a piece of copper wire that can distinguish between certain websites and only let certain ones work but not others or, they are really stupid at Sky and found a minor fault cos it was easy to test my line but cannot work out that there is a much more serious fault preventing me from accessing 90% of the worlds websites. And yes, I have tried all the obvious and not so obvious things...

[supremetwo]

the useless people at Sky say it will take up to 5 working days to fix as they say there is a fault near my house.

Is it Sky or BT Openreach?

I reported a Sky line fault this summer on a Sunday evening and BT Openreach turned up on Monday morning.

I suspect that there is a backlog of fault repairs due flooding just now.

[ianh64]

There was no fault. It was just an excuse by Sky so that they didn't need to get someone to fix things at the exchange.

Anyway, credit where credit is due. Its all back up and running now - and no engineer was needed to fix the non existent fault at my house. Just a shame they have to make it such hard work to convince them that 10+ devices in the house have not suddenly had their firewalls or parental controls turned on (including internet TV's trying to play BBC iPlayer) and the fact that I get some web sites working but not others (and the split is totally consistent, ie google.com always failed, google.co.uk always worked) and that my mobile phone (and mobile internet) works fine through the broadband VPN signal booster probably means that the fault does not lie in or near my house no matter what their check list tells them they must check.

Anyway, unfortunately PVoutput updates from my Pi/Pthon script has not restarted after the 12 hour outage and as I was running headless and without telnet enabled, I won't be able to ses what went wrong until I can connect it all back up again.