One wire pump sensing thoughts

[wyleu]

I wanted to spiral the sensors, so think yourself lucky we got away with what we show. It's really a test bed and as such it's something that's kind of nice to have done, and will be handy in some discussions about heating modes and other bits and pieces. Easy to do at the start not exactly an easy retro fit. As with every project I've reached the snowed under stage, and I'll get stuff up when I do it. I've got some 3D models of pumps and other such stuff to add, and that will require a view selection mechanism.

The default log in will probably look a bit like this ...
http://195.112.27.196:81/view/1/, so when you log on we establish this sort of ident for you. You specify an long and lat an elevation and an area of interest and thats what the glass sphere/cube thing is all about. You have to render something so that's about as simple as I've come up with at the moment that looks vaguely like some sort of building and doesn't tax the system too much.

 Then as you add elements, measurements and such like it will appear as active links like the air heater text link. If you float your mouse over the text box ou should find you hve a link to the modifiers section. Obviously more to follow in that department and if anyone is into designing a couple of small icons to represent graphs, modifiers and other bits and pieces It woul save me from indulging in an area I'm not too good at, The arrow buttons in the modifier section give you a rough idea of where my 2D design skills start and stop.


I can render up the entire python app as an exe, sure it's python but you only get the bits of the interpretor you need. It's all nicely packaged. You don't install python, you don't even realise it's involved. I'd publish the source and checksums for those that like that sort of thing.

The push XML to the server would be pretty much the way I'd be going.

[langstroth2]

We could get LogTemp to email (SMTP) the output to a specific navitron email address. The email server would extract file(s) automatically and sends where ever needed, admittedly more work at the server end, and depends what email server is running (ISP or own) as to how easy that would be.

Actually scratch that, now I look at my LogTemp install I think it only sends alarms via SMTP, not the standard XML output file.

[wyleu]

That's the conclusion I reached as well,

Automated E-mail is another area that seemed like a good idea at the time but in this day and age is subject to abuse.

If we do set up log-ins how would people want that to work? everything visible?, nothing visible to anyone except Sys Admin and yourself?, groups?, invite only?   ( you do publish the data yourself so ultimately you have control of what you send, and once again the idea of the script on your machine enforcing your own rules seems a bit more secure. Because you have the one-wire netwrok connected this can act as a fairly sophisticated dongle, so that would be a good place to control all your rules from).

We can store a fair bit, there is a database after all but quite how much history would people want? PAchube are being understandably fairly conservative in this regard, and I don't think we want to save every squeak coming out of every one-wire sensor from here to timbuktu, but it certainly would seem interesting to tie one-wire sensors down to locations, and that's not such an enormous lump of data.

[ericw]

While I agree that using a PIC with an 8MHz internal oscillator will reduce the component count I'm not sure it's fast enough.
The critical timing requires that you have clamped the bus low within a couple of microseconds of a negative edge.
I would have checked it but the local Maplin didn't stock 8 MHz crystals to use with my existing chip, so I think its a matter of suck it and see. The published Microchip master routines are much slower and so if you are using your own master rather than a DS9440 then you would probably be OK even at 4MHz.

I don't think there is much demand for the addition functions within the chip, even though they come virtually free, I was just trying to add what I considered useful additional functionality to the temperature plots of LogTemp. I did consider a multifunction chip (switch closure, A/D and counter) but decided that it would probably be more useful to pipe the bus around and have an single function interfaces near the point of use.

While LogTemp is a convenient and very easy to use program it does have its limitations and I would suggest it's forte is doing simple plots of a handful of curves and helping with diagnostics rather than bulk data logging.

Personally because the controlling input to the system (sunshine) is so variable and totally out of your control I cannot see much to be gained in publishing 'real time' data logging and would have thought that just uploading a complete graph periodically would be just as useful for information/comparison purposes. But then I'm not of the YouTube generation.

[kristen]

"I can render up the entire python app as an exe"

Do well!

"there is a database after all but quite how much history would people want?"

I wouldn't throw anything away - no telling what detail might be needed for retrospective analysis at some future date.

I think it might be interesting to convert adjacent time recording to "aggregated spans" where the values are the same (i.e. within some permitted tolerance).

Then a reading taken every, say, minute for an hour might be aggregated into a single record, spanning that hour, because the temperature had not changed by more than the permitted tolerance.

This method would, however, make reporting a nightmare (compared to having a continuous stream of values), so its only benefit is reduction of storage space.

It would be important that there were no gaps in the original data - if a sensors stops transmitting for 5 minutes, but then resumes at the previous value, that shouldn't be relied on and would need to be aggregated as two elapsed time records - the one before the gap, and the one after.

I expect that the odd missed reading could safely be ignored (depending on the interval between readings - readings every 10 seconds, and missing for 60 seconds, would be fine, but if the readings are only gathered once every 5 minutes then perhaps even a single missing reading should prevent aggregating that span?)

In a similar vein the odd spike could be ignored.  It is not feasible for the bottom sensor in my tank to change by 20 degrees, and back again, in the space of a minute - it means that I accidentally pulled the sensor out of the pocket and stuffed it back in as quickly as I could!! and that reading could be ignored (replaced by a straight line average between adjacent values would be reasonable). Needs some careful testing of scenarios where "spike suppression" would be permitted

[Paulh_Boats]

Chris,

The FTP account only needs write access to one folder with a maximum of 1Mb storage (no delete access and all other permissions removed). Then the data can be copied from that FTP folder to the folder that broadcasts HTML to the public by an administrator account.

The FTP account password would be given on a need to know basis to trusted forum members. The worst that could happen is that the FTP folder will fill up with 1Mb of junk, but it won't be visible to the public.

-Paul

[kristen]

Not entirely sure I agree. On our publicly facing servers we only install (well, "run") FTP if we absolutely have to, and the majority of them have 3rd party FTP which is further secured by SSL.  I'm not a systems guy, but I know that our systems folk went to considerably lengths to make sure the servers were secured from real threats (i.e. as distinct from potential threats).

But as I say my info is second hand.

[Paulh_Boats]

kristen,

I was thinking of an FTP server that cannot "see" the rest of the network. Then another server with read access to FTP copies the data on a regular basis. So the FTP server is 'incoming' only and not worth hacking because it only contains temperature logs. I'm assuming of course that we don't have a minute by minute live system, just a historical system.

AKA a DMZ:

http://en.wikipedia.org/wiki/Demilitarized_zone_(computing)

But if Navitron only have one server then the DMZ cannot exist.


Regarding the FTP accounts, yes it is time consuming but a lot easier than helping dozens of users set up their home routers as servers.


Of course another approach is to download XML data from my site and other personal sites and assemble it elsewhere. Then my ISP can worry about security:
http://www.millibee.com/logtemp/last.xml

-Paul

[sleepybubble]

I've been away too long, and I am a bit numptyish on some of this stuff.... But is the ultimate purpose of this project to build a box that hooks into a TDC-e that pushes solar gain data to a centralised (navitron) server?
What is the significance of the pump I/O state? like I say I've been off the radar for a while and the last major discussion I followed on this sort of vain involved folk trying to hack the TDC-e itself, I seem to rerember they were a bit unstable with external calls but some sort of workable solution came out of it. Has this approach been discounted as a way forward for some reason? perhaps security issues on end user networks. I only ask as I would presume the TDC-e as well as registering temperatures that are addressable would also register the pump relay state.
Or are you looking to achieve something far more generic and (cheap) that doesn't involve everybody have a TDC-e?
I'm only asking so I can understand why your all banging your neurons about...

[wyleu]

 I'm working on sucking data out of a TDC4-e at the moment.

The pump monitoring can, as you say, may be done that way, but there's a constant desire for general pump mains monitoring if it's only for underfloor heating and central heating for example.

[hiccup]

Ooh TDC4e? That's interesting - only knew about the 3e. Been a while since I looked at this stuff. Price is a bit scary but that includes a flow sensor so could be justifiable to SWMBO.

Any updated docs on the protocol?

Hic!

[wyleu]

I've got an interface layer constructed and can read sensor values, but I'm waiting to hear about one or two inconsistencies. The device only supports one connection at a time so any PC should probably act as a proxy to allow the supplied apps to work if required. The display proxy is reliable and accurate but involves a considerable ( well for a small device) amount of traffic, but it seems reliable and consistent. The other supplied app seems less complete, it seems to make an assumption about the protocol that isn't delivered by the box itself . Make sure you earth the pipework properly as the flow sensor is sensitive to mains hum and will if left unconnected on the bench read a consistent flow that confuses things. Still no sign of a reset or a firmware version available via the protocol.

[wyleu]

Re-reading this thread I noticed someone mentioned they couldn't see the pictures.

The Images are stored on the navitron showroom monitor laptop and as such needs a delay while it starts itself up, stretches, yawns once and then replies. So give it a bit of time if you want to see the images.