Data logging... how to.

[Mich]

I'm in a little project, trying to tackle a few problems.

I need to measure a temperature, and store it for later readout.
I will be using an AVR for temp. reading, and doing some other stuff.
The temperature will need to be stored in hourly samples, with a runningtime
for about nearly a year.

In my idea i would be storing hour-day-month-year-temp. I calculated that a
needed about 4 bytes for this, so for nearly a year, i'd be oke with 256Kx8,
which i prolly need to do external from the AVR.
Now, after the year, the data will be shifted continously , so there will
always be the latest data reading about 12 months back.

Questions:
- Am i going the right route with simply storing in hhddmmyytt, or am i
wasting ram space?
- Would i need to shift every time all 256Kbytes one adress down, and place
the latest value on top, or would a 'current pointer' do the job, which
could be used in the MSB from the 4 bytes, which is prolly not used.

Any help appriciated.


Thankx,

Mich

 

[Andrew Holme]

Mich wrote:

I'm in a little project, trying to tackle a few problems.

I need to measure a temperature, and store it for later readout.
I will be using an AVR for temp. reading, and doing some other stuff.
The temperature will need to be stored in hourly samples, with a
runningtime for about nearly a year.

In my idea i would be storing hour-day-month-year-temp. I calculated
that a needed about 4 bytes for this, so for nearly a year, i'd be
oke with 256Kx8, which i prolly need to do external from the AVR.
Now, after the year, the data will be shifted continously , so there
will always be the latest data reading about 12 months back.

Questions:
- Am i going the right route with simply storing in hhddmmyytt, or am
i wasting ram space?

If the samples are taken at regular hourly intervals, you don't need to
store the timestamp, at least not on *every* sample - you can work it out
afterwards from the position.

- Would i need to shift every time all 256Kbytes one adress down, and
place the latest value on top, or would a 'current pointer' do the
job, which could be used in the MSB from the 4 bytes, which is
prolly not used.

Niether - use a "circular list" - when you get to the bottom, start
over-writing again from the top. You don't need space for a flag against
every sample - your "pointer" should be stored *outside* the list. The
pointer tells you where the last sample is. If you know when the last
sample was taken, you can work back to find the temperature at any time in
the previous 12 months.

 

[Mich]

"Luhan Monat" <x@y.z> wrote in message
news:66The.81154$tQ.50961@fed1read06...

Mich wrote:
I'm in a little project, trying to tackle a few problems.

I need to measure a temperature, and store it for later readout.
I will be using an AVR for temp. reading, and doing some other stuff.
The temperature will need to be stored in hourly samples, with a
runningtime
for about nearly a year.

In my idea i would be storing hour-day-month-year-temp. I calculated
that a
needed about 4 bytes for this, so for nearly a year, i'd be oke with
256Kx8,
which i prolly need to do external from the AVR.
Now, after the year, the data will be shifted continously , so there
will
always be the latest data reading about 12 months back.

Questions:
- Am i going the right route with simply storing in hhddmmyytt, or am i
wasting ram space?
- Would i need to shift every time all 256Kbytes one adress down, and
place
the latest value on top, or would a 'current pointer' do the job, which
could be used in the MSB from the 4 bytes, which is prolly not used.

Any help appriciated.


Thankx,

Mich




Hi,

Here's what I did for storing a lot of sample data...

http://members.cox.net/berniekm/spydar.html

Basically, only store delta times. I needed to know the absolute time,
so I do that with the program that reads the data from the unit.

Some of this may apply to you.

--
Luhan Monat: luhanis(at)yahoo(dot)com
http://members.cox.net/berniekm
"Any sufficiently advanced magick is
indistinguishable from technology."

Thankx Andrew, Pertus and Luhan.

Though your ideas might not be completely suitable for my application, it
made me think about it a bit more, and some bits might be useable. If not
used, i learned something again.

Thankx again,

Mich

 

[John Fields]

On Mon, 16 May 2005 22:41:40 +0200, "Mich" <michelpl.nocrap@xs4all.nl>
wrote:

Thankx Andrew, Pertus and Luhan.

Though your ideas might not be completely suitable for my application, it
made me think about it a bit more, and some bits might be useable. If not
used, i learned something again.

---
If you know that your readings are going to be an hour apart, then as
everyone else has already stated, once you start the thing its fate is
sealed.

One thing that I think wasn't adequately addressed was the transition
from year one to year two. That is, with a single time stamp at the
beginning of year one and another at the beginning of year two, I
don't believe there's a way to differentiate between the two sets of
data after the second time stamp if there isn't something impressed on
the data from year two to differentiate it from the year one data.
That is, with no markers, how could one tell what was overwritten and
what wasn't until year three's time stamp was reached.

If it's only a two year deployment, then a single bit of memory could
be added to the data stream dynamically without having to pay for
extra storage. When year one ends, toggle an RS latch and include
that new state as part of the data word. If it's year one data the
year bit will be zero, but if it's year two data the year bit will be
one.

--
John Fields
Professional Circuit Designer

 

[John Fields]

On Mon, 16 May 2005 17:05:39 -0500, John Fields
<jfields@austininstruments.com> wrote:

On Mon, 16 May 2005 22:41:40 +0200, "Mich" <michelpl.nocrap@xs4all.nl
wrote:


Thankx Andrew, Pertus and Luhan.

Though your ideas might not be completely suitable for my application, it
made me think about it a bit more, and some bits might be useable. If not
used, i learned something again.

---
If you know that your readings are going to be an hour apart, then as
everyone else has already stated, once you start the thing its fate is
sealed.

One thing that I think wasn't adequately addressed was the transition
from year one to year two. That is, with a single time stamp at the
beginning of year one and another at the beginning of year two, I
don't believe there's a way to differentiate between the two sets of
data after the second time stamp if there isn't something impressed on
the data from year two to differentiate it from the year one data.
That is, with no markers, how could one tell what was overwritten and
what wasn't until year three's time stamp was reached.

If it's only a two year deployment, then a single bit of memory could
be added to the data stream dynamically without having to pay for
extra storage. When year one ends, toggle an RS latch and include
that new state as part of the data word. If it's year one data the
year bit will be zero, but if it's year two data the year bit will be
one.

---
Correction: There would be one bit of overhead per byte (8766 bits
total) for storage of the year bit.


--
John Fields
Professional Circuit Designer

 

[Luhan Monat]

Mich wrote:

I'm in a little project, trying to tackle a few problems.

I need to measure a temperature, and store it for later readout.
I will be using an AVR for temp. reading, and doing some other stuff.
The temperature will need to be stored in hourly samples, with a runningtime
for about nearly a year.

In my idea i would be storing hour-day-month-year-temp. I calculated that a
needed about 4 bytes for this, so for nearly a year, i'd be oke with 256Kx8,
which i prolly need to do external from the AVR.
Now, after the year, the data will be shifted continously , so there will
always be the latest data reading about 12 months back.

Questions:
- Am i going the right route with simply storing in hhddmmyytt, or am i
wasting ram space?
- Would i need to shift every time all 256Kbytes one adress down, and place
the latest value on top, or would a 'current pointer' do the job, which
could be used in the MSB from the 4 bytes, which is prolly not used.

Any help appriciated.


Thankx,

Mich

Hi,

Here's what I did for storing a lot of sample data...

http://members.cox.net/berniekm/spydar.html

Basically, only store delta times. I needed to know the absolute time,
so I do that with the program that reads the data from the unit.

Some of this may apply to you.

--
Luhan Monat: luhanis(at)yahoo(dot)com
http://members.cox.net/berniekm
"Any sufficiently advanced magick is
indistinguishable from technology."

 

[Mich]

"John Fields" <jfields@austininstruments.com> wrote in message
news:6ink815ss6204vi0lpnpc1fu531clhmfo6@4ax.com...

On Mon, 16 May 2005 17:05:39 -0500, John Fields
jfields@austininstruments.com> wrote:

On Mon, 16 May 2005 22:41:40 +0200, "Mich" <michelpl.nocrap@xs4all.nl
wrote:


Thankx Andrew, Pertus and Luhan.

Though your ideas might not be completely suitable for my application,
it
made me think about it a bit more, and some bits might be useable. If
not
used, i learned something again.

---
If you know that your readings are going to be an hour apart, then as
everyone else has already stated, once you start the thing its fate is
sealed.

One thing that I think wasn't adequately addressed was the transition
from year one to year two. That is, with a single time stamp at the
beginning of year one and another at the beginning of year two, I
don't believe there's a way to differentiate between the two sets of
data after the second time stamp if there isn't something impressed on
the data from year two to differentiate it from the year one data.
That is, with no markers, how could one tell what was overwritten and
what wasn't until year three's time stamp was reached.

If it's only a two year deployment, then a single bit of memory could
be added to the data stream dynamically without having to pay for
extra storage. When year one ends, toggle an RS latch and include
that new state as part of the data word. If it's year one data the
year bit will be zero, but if it's year two data the year bit will be
one.

---
Correction: There would be one bit of overhead per byte (8766 bits
total) for storage of the year bit.


--
John Fields
Professional Circuit Designer

Thank you John for your reply.
The year bit would surely make a difference.

Thankx again,

Mich