navitron
 
Renewable Energy and Sustainability Forum
UK's most popular Renewable Energy Forum May 23, 2012, 04:13:59 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
News: Anyone wishing to register as a new member on the forum is strongly recommended to use a "proper" email address - following recent spam/hack attempts on the forum, all security is set to "high", and "disposable" email addresses like Gmail, Yahoo and Hotmail tend to be viewed with suspicion, and the application rejected if there is any doubt whatsoever
 
Recent Articles: UPDATE ON DECC APPLICATION FOR LEAVE TO APPEAL TO THE SUPREME COURT | Yingli Green Energy's PV Module Ranks No.2 in TUV Rheinland Energy Yield Test | Navitron Solar Showers at Glastonbury for Year 5!
   Home   Help Search Login Register  
Pages: [1]   Go Down
  Print  
Author Topic: A standard solar thermal database.  (Read 1401 times)
desperate
Guest
« on: October 27, 2010, 05:32:38 PM »

As some of you may know StB has bravely agreed to help me set up a monitoring/logging system here at cactusville, so I figure to be able to start posting up some data once we have ironed out any bugs. This led me to wonder weather there is any value to us having some kind of spreadsheet on the forum here where we can all enter our data in a standardized style.

It is interesting to see all the different results posted by you all, but to make a direct comparison is quite fiddly, whereas if we could all enter data onto a single XL doc/graph or similar, provided there were say less than 20 plots,  would it be quite a good comparison?

I have no idea how practical this is, just a thought really.

Desperate
Logged
StBarnabas
Hero Member
*****
Offline Offline

Posts: 2111


St Barnabas Chapel (2009)


« Reply #1 on: October 27, 2010, 06:46:33 PM »

Desp
Wyleu was working on something like this but he has been very busy with the day job recently. Not sure if there has been any progress..
Sean
Logged


Gestis Censere. 40x47mm DHW with TDC3. 3kW ASHP, 9kW GSHP, 3kW Navitron PV with Platinum 3100S GTI, 6.5kW WBS, 5 chickens. FMY 2009.
wyleu
Guest
« Reply #2 on: October 27, 2010, 07:57:21 PM »

The important information is the context around the reading.
In database terms we're talking about how we structure the database.

It would be tempting to go all excel at the start of this and stick it all in one big table...
but it really needs breaking down.

say a sensor type table...

class Sensor(models.Model):
    name  =    models.CharField(max_length=255, help_text= 'A DS18B20')
    desc   =    models.TextField(blank = True)
    created_date = models.DateTimeField(auto_add = True)
    modified_date = models.DateTimeField(auto_add_now = True)

That allows us to not even care too much if the values are from one-wire or anything else.
You could add a type field as a foreign key but relevant data will have different structures to encompass and that is a bit of a pain in database terms because your generating different table structures dependent on the data, which means your going to have lots of fun combining and appending data sets.  KISS.

Now since we've got sensors we have units which are abstractions from our data value. It might be real data or it might be a spurious reading (85C) you can't really say at this point cos your just data gathering. Only later consideration can decide if it can be modified or indeed deleted.

class Unit(models.Model):
    name  =    models.CharField(max_length=255, help_text= 'Degrees Celsius')
    disp    =    models.CharField(max_length=255, help_text='°C')
    desc   =    models.TextField(blank = True)
    created_date = models.DateTimeField(auto_add = True)
    modified_date = models.DateTimeField(auto_add_now = True)
    content_object = generic.GenericForeignKey('content_type', 'object_id')

... Tea Time, I'll be back...

    

    
class Reading(models.Model):
    sensor = models.ForeignKey(Sensor)
    created_date = models.DateTimeField(auto_add = True)
    modified_date = models.DateTimeField(auto_add_now = True)


 it 'could' really be anything, and also might have some concept as to what was received and what you believe it's representing.

    unit = models.ForeignKey('Unit')
Logged
desperate
Guest
« Reply #3 on: October 27, 2010, 08:12:56 PM »

Err uummm, thanks for that Wyleu, I didn,t really understand any of that I,m afraid.

I guess you are thinking in terms of downloading data direct from the sensing electronics/computer? all well and good for you folks who understand it all. Maybe it would be possible to also have a simpleton version for myself and maybe others to manually input readings and times etc, which while not as elegant is doable in smallish quantities, but if it standardises a fair selection of results from our systems would still be a usefull resource.

I,m sort of guessing here, in case you couldn,t tell Grin

My given name is Dicky and I come from..........................Cactusville

Desperate
« Last Edit: October 27, 2010, 08:15:07 PM by desperate » Logged
wyleu
Guest
« Reply #4 on: October 27, 2010, 09:49:49 PM »

I'll fill the rest of that in when I have a little more time.

It's basically django www.djangoproject.com, which is a really good way of doing database stuff without writing very much...

You need something to read data from your sensors into database tables. To define all the elements efficiently you need to break it all down into lumps of related elements called tables. There are relationships between tables, so you can define a type of sensor in a table. You then define actual sensors ' The one in the bathroom ', that kind of thing  and then define a reading that is an example  of a type of  sensor and also adds the time the reading was taken and the value that was actually stored...
But the reading could be of several different types ( Integer  45, Decimal 45.3, Float 4.53e4,) it all depends on the kind of sensor so you need a mechanism to deal with this. dynamically typed languages ( like python) don't care about this sort of thing, value can be a float one minute and a True/False value the next. Languages like C are a bit fussier about that sort of thing...

The reading is done by a small loop of code, that looks something like...

.... setup stuff ....

sensors = owfs.sensors()

while(True):
    sensor_list = sensors.get_list_of current_sensors()
    for sensor in sensor_list:
        sensor.value = sensor.read_value()
        sensor.save()    

This runs for ever( True is always true so while never leaves the loop unless there is an error... )
And while this runs your database will fill up with lots of sensor values...

What you do with them then is a whole different ball game.




« Last Edit: October 27, 2010, 09:59:31 PM by wyleu » Logged
wyleu
Guest
« Reply #5 on: November 09, 2010, 06:22:10 PM »

I thought I'd better come back and try and make some sense on this thread, so here goes.

Seemingly you really need to record three bits of information for each reading.

Which Sensor produced the reading. This needs to be unique to avoid confusion and that's exactly the kind of things database's ( databi ?) like to do. In the One  world it's a no brainer to use the ID for this field and since you ain't going to do maths on it, make it a Character Field. You don't loose overhead in making this big in modern databases so make it 255 to keep it just below the point at which most databi loose the ability to index on it.

name  =    models.CharField(max_length=255, help_text= 'The sensor name')

so for a DS18S20 that would start with 10h and the DS18B20 would start with 28h. Now these are actually hexadecimal values (Base 16; you count 0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F. Computers like it cos it fit's into ones and noughts really easily so two characters represent all the numbers from 0 (00h) via 16 (10h) and 40 (28h)  up to 255 (FFh)  but since you ain't going to do any maths on them ( would you, for instance, ever need to multiply your house number by anything to any non frivolous purpose ...?) you don't wory two much about that and you just treat it is a chunk of text.

Next we need to know the time of the reading.

created_date = models.DateTimeField(auto_add = True)

Databai, generally have something that can represent a date and a time. There is some degree of standardization but moving dates and times between systems ( POSTGRES-> MySQL for instance ) can be a real pain. If it's done properly you should record information like time zone and from there onto geographic stuff but for most home systems the native date/time format will work. Just make sure you know about daylight saving times otherwise your going to find yourself comparing different noons and you'll have people in forums laughing at you behind your back, not buying you drinks or questioning anything you may say about your measurements.

and then the value...

and this is where it get's a little dependent on opinion.
There are several different field type for storing 'values', and different sensors might want to store different types of value.

Let's start with our old friend the DS18B20 & DS18S20. http://www.tempsensornews.com/generic-temp-sensors/comparison-of-the-ds18b20-and-ds18s20-1-wire%C2%AE-digital-thermometers/ Seemingly they are identical, and by and large will do the same sorts of things. Except one of them you can send values to it to change the resolution it can offer.That aspect alone instantly seperates the two of them conceptually because to get an accurate reading that you can believe requires you know other bits of information.
However they do present data identically, to quote from the datasheets...
The temperature data is stored as a 16-bit sign-extended two’s complement number

So one could store precisely that if your database has that kind of definition and that would be the end of the issue, but for one small aspect. We are much more likely to want to actual store it as a number we can manipulate as a number, which we happen to know corresponds to a temperature.
We could store it as an integer...

so 20 °C would be stored as the number 20. This gets useful cos you can now do maths like things like highest temperature or lowest temperature or averages.
But that isn't quite as accurate as it could be.

So the next choice is the engineers favourite the float...

value = models.FloatField()
The database stores two numbers which are a number in binary raised to a power by anothe number. The problem her is that accuracy can get interesting. You very quickly get issues like 2.0 * 2.0 = 3.99999999999999975. Which whilst actually at an accuracy way above anything we might choose to measure till tends to confuse, especially at the display end.

There is the decimal,  20.375

value = models.DecimalField(..., max_digits=5, decimal_places=2)

This does at least keep everything in a nicely handled format without all the fun of displaying floats, so that might be a good solution and from the software side it is exactly accurate.

At the end of the day it's up to you cos your publishing the data.

Of course for the half-bee like projects that use pseudo thermometers as switches you always have the Boolean field, which allows you to show on and off ness and nothing else.

value = models.BooleanField(default = True)

various other types exist, heres' the django take on it http://docs.djangoproject.com/en/1.2/ref/models/fields/#integerfield but any database supports these basic concepts.

So as you can see a Standard Solar thermal database really needs a lot more detail about Sensors before it can really be a standard. The context of the database is very important and really a standard data base should be able to include suh items as ImageFields to contain pictures of the sensors, and geographical data to actually say where it was.

It gets quite big quite quickly. These are obviously just my thoughts. I'd love to hear anyone else's view, especially if the disagree  extrahappy

More available if people want it. facepalm










Logged
Countrypaul
Sr. Member
****
Offline Offline

Posts: 336


« Reply #6 on: November 09, 2010, 06:49:08 PM »

>> ( would you, for instance, ever need to multiply your house number by anything to any non frivolous purpose ...?)

Multiply your house number by 10 to get the number of metres from the start of the road - in certain parts of New Zealand  extrahappy

Paul
Logged
desperate
Guest
« Reply #7 on: November 09, 2010, 07:15:32 PM »

Ok thanks Wyleu, obviously it is a lot more complicated than I realised, I guess I was thinking of something like that IMeasure database but tailored to suit as lot here.

I am afraid in this subject I dont really know what I don't know, what I do know though, is that there is a lot that I don't know.


Desperate
Logged
StBarnabas
Hero Member
*****
Offline Offline

Posts: 2111


St Barnabas Chapel (2009)


« Reply #8 on: November 09, 2010, 08:35:13 PM »

Desp
can chat tomorrow. As of c 13:30 Desp will be in residence at StBC!
Logged


Gestis Censere. 40x47mm DHW with TDC3. 3kW ASHP, 9kW GSHP, 3kW Navitron PV with Platinum 3100S GTI, 6.5kW WBS, 5 chickens. FMY 2009.
wyleu
Guest
« Reply #9 on: November 10, 2010, 09:02:40 PM »

A more suitable concept might be http://www.pachube.com/

The free account is limited but would, standardise a lot of elements. So rather than develop data straight into a highly personalized database you could send it as an information packet to pachube and then produce a database proxy that could 'share' the data stream.

Obviously I'd do it all pythonically http://community.pachube.com/python but as ou can see lots of alternative methods of sinking and sourcing data exist...





Logged
desperate
Guest
« Reply #10 on: November 12, 2010, 12:21:56 AM »

Hey whaddaya know,  while doing  a bit of plumbery at StBs lovely home, in return he's setting me up with a one wire and Arduino system with datalogging capacity. So Wyleu, Guru status just might be a possibility after all Grin

Yehhaaww

Desp
Logged
Pages: [1]   Go Up
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.16 | SMF © 2011, Simple Machines Valid XHTML 1.0! Valid CSS!