SMA Webbox XML Problems

[scfnet]

Does anyone have a SMA webbox and uses the XML data feeds from it?

I have two issues,

The first is probably not an issue with the webbox but I can't figure it out. If I configure a local ftp server to be the recipient for push data, the zip file (contains the last three xml files) is corrupt. The content record is wrong and only the first xml file can be extracted. It only appears to be an issue with the pureftpd on linux, switching to vsftpd made the issue go away but I would really like to know why.

The second is the data itself. The zip files have a filename which is the date of the last xml file in the zip file in format YYYY-MM-DD_HHmmss.zip and the same is true for the xml files. The times here are in 24 hour format. However inside the zip file the timestamp is in 12 hour format with no indicator of the AM / PM.

E.g.

File 2011-06-12_051500.xml

Code:

<WebBox>
    <Info>
        <Created>2011-06-12T05:15:00</Created>
        <Culture>en</Culture>
        <UtcOffset>0</UtcOffset>
    </Info>
    <CurrentPublic>
        <Key>SN: 2100437728 (SN: 2100437728):2100437728:Total yield</Key>
        <Mean>39.271</Mean>
        <Base>1</Base>
        <Period>300</Period>
        <Timestamp>2011-06-12T05:15:00</Timestamp>
    </CurrentPublic>
</Webbox>

File 2011-06-12_213000.xml

Code:

<WebBox>
    <Info>
        <Created>2011-06-12T09:30:00</Created>
        <Culture>en</Culture>
        <UtcOffset>0</UtcOffset>
    </Info>
    <CurrentPublic>
        <Key>Sunny WebBox:155002330:Total yield</Key>
        <Mean>47.211</Mean>
        <Base>1</Base>
        <Period>300</Period>
        <Timestamp>2011-06-12T09:30:00</Timestamp>
    </CurrentPublic>
</Webbox>

Has any one seen this before?  I have looked at all the settings but can't find anything relevant.

I have a ticket open with SMA since monday but they have not responded yet.

Thanks

Alastair

[EccentricAnomaly]

It only appears to be an issue with the pureftpd on linux, switching to vsftpd made the issue go away but I would really like to know why.

For funny file corruption with FTP the first place I'd look would be Binary/ASCII mode. In Binary mode files are transferred byte for byte. In ASCII mode line ends are sent in some canonical form (I forget what but suspect it's a single line feed character) and translated from/to the local convention (carriage return/line feed on Windows, line feed on Un*x). An ASCII mode transfer of a file containing pure binary data can result in a horrible mess.

Even if there happen to be no bytes in your ZIP file headers which match line end characters conversion of line ends in the data would mean that lengths don't match which might account for being able to extract just the first file.

Defaults for the mode vary and most FTP software has lists of file extensions with different defaults. It could be differences between these that account for the different behaviour you see with the servers you tried.

Sorry, no idea about the times.

[scfnet]

That would make sense.

Looking at the docs for pure-ftpd it is not possible to force it to use binary mode unless you recompile it and the webbox does not send the bin command.

I'm still not sure about the vsftp daemon but it appears to work so I think I will just leave that running. If it works don't touch!

Anyone got any ideas about the xml file format, or have a contact at SMA? I have still not had any response.

Alastair