CMOS camera w/ USB2 -- crazy?

[GB]

Hi,

I'm a firmware guy pulled into a project well out of my area of
expertise. My boss wants to build (essentially) a digital camera
using an image sensor chip (1600x1200) and output it's data
"as fast as possible" using USB2.0. His initial concept, being
that I'm a firmware guy, was to use a "really fast micro."
Normally the company is involved with small 8-bitter micro
projects, so you can see I'm well out of my normal bounds.

Now this seems like a pretty big stretch to me... and I don't see
how it can be done without the speed of hardware (the image
chips all seem to have clock speeds in the tens of MHz and the
USB2 transfer rate is 480Mbps (theor.). Do aspects of this
project require an FPGA to keep the data "as fast as possible?"
If we use and FPGA for camera-to-RAM and then use a
micro for the USB2 part, what kind of fast micros can
move data at that rate?

Also, this is something that I am sure we will have to contract
out, so if you have any past experience with this, please let
me know your thoughts (and if you are available).

Thanks!

 

[rickman]

GB wrote:

Hi,

I'm a firmware guy pulled into a project well out of my area of
expertise. My boss wants to build (essentially) a digital camera
using an image sensor chip (1600x1200) and output it's data
"as fast as possible" using USB2.0. His initial concept, being
that I'm a firmware guy, was to use a "really fast micro."
Normally the company is involved with small 8-bitter micro
projects, so you can see I'm well out of my normal bounds.

Now this seems like a pretty big stretch to me... and I don't see
how it can be done without the speed of hardware (the image
chips all seem to have clock speeds in the tens of MHz and the
USB2 transfer rate is 480Mbps (theor.). Do aspects of this
project require an FPGA to keep the data "as fast as possible?"
If we use and FPGA for camera-to-RAM and then use a
micro for the USB2 part, what kind of fast micros can
move data at that rate?

I don't think the micro should be in the data path. A micro can be used
for control and management, but the data path should be pure hardware
for maximum throughput. If you look around, you will find that there
are USB chips with integrated micros (many are 8051s). But the USB 2.0
chips typically have a DMA engine for the data path.

Also, this is something that I am sure we will have to contract
out, so if you have any past experience with this, please let
me know your thoughts (and if you are available).

We have some background in this area. If you are interested, please
contact me at the email address below or at sales (at) arius.com.

--

Rick "rickman" Collins

rick.collins@XYarius.com
Ignore the reply address. To email me use the above address with the XY
removed.

Arius - A Signal Processing Solutions Company
Specializing in DSP and FPGA design URL http://www.arius.com
4 King Ave 301-682-7772 Voice
Frederick, MD 21701-3110 301-682-7666 FAX

 

[hamilton]

What image sensor chip are you looking at ???

GB wrote:

Hi,

I'm a firmware guy pulled into a project well out of my area of
expertise. My boss wants to build (essentially) a digital camera
using an image sensor chip (1600x1200) and output it's data
"as fast as possible" using USB2.0. His initial concept, being
that I'm a firmware guy, was to use a "really fast micro."
Normally the company is involved with small 8-bitter micro
projects, so you can see I'm well out of my normal bounds.

Now this seems like a pretty big stretch to me... and I don't see
how it can be done without the speed of hardware (the image
chips all seem to have clock speeds in the tens of MHz and the
USB2 transfer rate is 480Mbps (theor.). Do aspects of this
project require an FPGA to keep the data "as fast as possible?"
If we use and FPGA for camera-to-RAM and then use a
micro for the USB2 part, what kind of fast micros can
move data at that rate?

Also, this is something that I am sure we will have to contract
out, so if you have any past experience with this, please let
me know your thoughts (and if you are available).

Thanks!

 

[Phil Hays]

GB wrote:

I'm a firmware guy pulled into a project well out of my area of
expertise. My boss wants to build (essentially) a digital camera
using an image sensor chip (1600x1200) and output it's data
"as fast as possible" using USB2.0. His initial concept, being
that I'm a firmware guy, was to use a "really fast micro."
Normally the company is involved with small 8-bitter micro
projects, so you can see I'm well out of my normal bounds.

Now this seems like a pretty big stretch to me... and I don't see
how it can be done without the speed of hardware (the image
chips all seem to have clock speeds in the tens of MHz and the
USB2 transfer rate is 480Mbps (theor.). Do aspects of this
project require an FPGA to keep the data "as fast as possible?"
If we use and FPGA for camera-to-RAM and then use a
micro for the USB2 part, what kind of fast micros can
move data at that rate?

Also, this is something that I am sure we will have to contract
out, so if you have any past experience with this, please let
me know your thoughts (and if you are available).

Sounds like a good project for an FPGA. Micros don't keep up with this kind of data rates, standard parts do what everyone else is doing (and this doesn't sound standard), and ASICS can do this, if you have the volume to make it worthwhile, but it would be wise to prototype the design in a FPGA first.

I've done several FPGA designs to solve similar problems. I'm not available, but I'm fairly sure I know of someone fairly good that is available, and if you send mail to (attbi.com phil-hays) I'll get you in contact with him.


--
Phil Hays

 

[GB]

"hamilton" <hamilton@deminsional.com> wrote in message
news:3f5b561c_3@omega.dimensional.com...

What image sensor chip are you looking at ???

That's another undecided at this time, but CMOS most likely.

GB

 

[GB]

"rickman" <spamgoeshere4@yahoo.com> wrote in message
news:3F5B4FAC.41D84F7B@yahoo.com...

I don't think the micro should be in the data path. A micro can be used
for control and management, but the data path should be pure hardware
for maximum throughput. If you look around, you will find that there
are USB chips with integrated micros (many are 8051s). But the USB 2.0
chips typically have a DMA engine for the data path.

From what I understand, the image size is 1-2 MB (only need 8-bit depth)
large and the USB packets are quite small in comparison. So there needs to
be some smarts for setting up what part of the data gets DMA'd to the
USB endpoint buffers for any given transfer? Or does the FPGA
implementation just point to the image data stored in RAM (assuming image
frame captured to some local RAM buffer) and never move the data from
image_buffer -> to_endpoint -> to_USB (essentially skipping the middle
man by just using a pointer of sorts)?

It's just a still picture (not a movie). Doesn't this seem like a somewhat
common function -- meaning wouldn't there be standard chips or
implementations that do this?

Thanks again,
GB

 

[Mario Trams]

GB wrote:

It's just a still picture (not a movie).

Aha, that's an important fact as it greatly reduces the requirements.
Depending on what "as fast as possible" (in terms of latency) means,
a DSP or a better RISC microprocessor might be a better solution
for you. At least you won't have to introduce yourself to the new
(for you) area of FPGAs.

I don't know how the interface of the CCD chip you are planning to
use is working, but I can imagine that it uses some standard interface
that can be found on higher-class processors.

So I suggest you to check some DSPs first (Analog Devices, Texas
Instruments, Motorola, ...). Although you might not need the computing
power they provide, perhaps you can use their hardware interfaces.
DSPs are often used for image processing and therefore have a close
relationship to CCDs.

Perhaps you could also ask for this issue in comp.dsp.

Regards,
Mario

--
----------------------------------------------------------------------
Digital Force / Mario Trams Mario.Trams@informatik.tu-chemnitz.de
Mario.Trams@wooden-technology.de
Chemnitz University of Technology http://www.tu-chemnitz.de/~mtr
Dept. of Computer Science Tel.: (+49) 371 531 1660
Chair of Computer Architecture Fax.: (+49) 371 531 1818
----------------------------------------------------------------------

 

[Jim Granville]

GB wrote:

"rickman" <spamgoeshere4@yahoo.com> wrote in message
news:3F5B4FAC.41D84F7B@yahoo.com...

I don't think the micro should be in the data path. A micro can be used
for control and management, but the data path should be pure hardware
for maximum throughput. If you look around, you will find that there
are USB chips with integrated micros (many are 8051s). But the USB 2.0
chips typically have a DMA engine for the data path.

From what I understand, the image size is 1-2 MB (only need 8-bit depth)
large and the USB packets are quite small in comparison. So there needs to
be some smarts for setting up what part of the data gets DMA'd to the
USB endpoint buffers for any given transfer? Or does the FPGA
implementation just point to the image data stored in RAM (assuming image
frame captured to some local RAM buffer) and never move the data from
image_buffer -> to_endpoint -> to_USB (essentially skipping the middle
man by just using a pointer of sorts)?

It's just a still picture (not a movie). Doesn't this seem like a somewhat
common function -- meaning wouldn't there be standard chips or
implementations that do this?

Yes, Data-> USB 2.0 is common, and a good example of a std chip
is the Cypress chip someone has already mentioned.

You may need a CPLD or small FPGA to run the CMOS sensor
clocks, and help the transfer, but the idea is to do as much in
the standard uC.USB2.0 package as you can.

I would also nail down just what "as fast as possible" really
means. Tell your boss there is a speed/price/development time
tradeoff, and find what speed the product really NEEDS.

Also find out, early on, if any form of compression is
envisioned, or if a simple 'frame dump' is fine.

-jg

 

[Khim Bittle]

On Sun, 07 Sep 2003 15:03:39 GMT, "GB" <donotspam_grantbt@jps.net>
wrote:

Hi,

I'm a firmware guy pulled into a project well out of my area of
expertise. My boss wants to build (essentially) a digital camera
using an image sensor chip (1600x1200) and output it's data
"as fast as possible" using USB2.0. His initial concept, being
that I'm a firmware guy, was to use a "really fast micro."
Normally the company is involved with small 8-bitter micro
projects, so you can see I'm well out of my normal bounds.

Now this seems like a pretty big stretch to me... and I don't see
how it can be done without the speed of hardware (the image
chips all seem to have clock speeds in the tens of MHz and the
USB2 transfer rate is 480Mbps (theor.). Do aspects of this
project require an FPGA to keep the data "as fast as possible?"
If we use and FPGA for camera-to-RAM and then use a
micro for the USB2 part, what kind of fast micros can
move data at that rate?

Also, this is something that I am sure we will have to contract
out, so if you have any past experience with this, please let
me know your thoughts (and if you are available).

Thanks!

Hello :

The first question that comes to mind is why are you designing this ?
( other than your boss told you to )

One answer may be high volume production where an absolutely cost
optimized solution is the correct answer.

Another answer may be special requirements for your application. If
this is so then you have not provided enough information above to tell
us what these special requirements are to suggest an architecture.

I ask this question simply because there are numerous small oem camera
modules on the market with hi res cmos sensors and USB2 output that
can get you to market rapidly and become cost effective as volumes
increase. An example may be found at
http://www.lumenera.com/oem.htm

If we proceed along the path that you need to design such a module :

Several companies are coming out with single chip imaging controllers
( national ,etc ) but I am not aware of an available part to do
exactly what you are looking for ... but I would do a very careful
search as this is the most streamlined approach.

An FPGA may be a good solution to interconnect the sensor module ,
capture memory and the USB controller. The controller ( Cypress )
will normally setup the USB endpoints etc. and then get out of the
way. The FPGA can handle all of the high speed timing and perhaps a
SDRAM controller.

Another most intriguing solution (hey this is the comp.arch.fpga
group) , however the most engineering intensive ($) , would be to use
an FPGA with softcore processor and perhaps a USB core ... so the
entire solution except sensor, memory and USB physical drive chip
could be in the FPGA. The softcore processor could be programmed in C
and code changes simply bootloaded in to a flash memory by the
softcore processor without an FPGA design change ( nice when you have
contracted the FPGA design out of house yet the in house programmers
can write C code which runs on the processor in the FPGA ). ( I am
currently involved with a video design utilizing Altera Nios softcore
processor / Cyclone FPGA designs )

Good Luck, check out the link below, we have significant video and
imaging experience and may be able to assist your design efforts.

Khim Bittle
khimbittle@cliftonREMOVEsystems.com
(remove REMOVE from email address to respond)

Clifton Systems Inc.
http://www.cliftonsystems.com/design

 

[john jakson]

"GB" <donotspam_grantbt@jps.net> wrote in message news:<GFJ6b.2610$PE6.1782@newsread3.news.pas.earthlink.net>...

"hamilton" <hamilton@deminsional.com> wrote in message
news:3f5b561c_3@omega.dimensional.com...

What image sensor chip are you looking at ???


That's another undecided at this time, but CMOS most likely.

GB

Until recently I was under the impression that CMOS sensors were junk
and were of low resolution & quality typically 320.240 used in $0-50
cameras. I have an old Connectix webcam device thats is all green that
demonstrates that.

Recently though a Micron engineer persuaded me that CMOS is looking
very much up and showed me some upcoming sensors for both high
resolution & higher quality still not quite as good as CCD but far
cheaper to make & potentially far easy to integrate with externl
interfaces. If the quality is there, I would expect USB2/FW to end up
on chip.

I will wait and see if they show up in the mid range cameras coming
soon.

But check out their imaging division for specs if you care.

 

[Magnus Homann]

"GB" <donotspam_grantbt@jps.net> writes:

"rickman" <spamgoeshere4@yahoo.com> wrote in message
news:3F5B4FAC.41D84F7B@yahoo.com...

I don't think the micro should be in the data path. A micro can be used
for control and management, but the data path should be pure hardware
for maximum throughput. If you look around, you will find that there
are USB chips with integrated micros (many are 8051s). But the USB 2.0
chips typically have a DMA engine for the data path.

From what I understand, the image size is 1-2 MB (only need 8-bit depth)
large and the USB packets are quite small in comparison. So there needs to
be some smarts for setting up what part of the data gets DMA'd to the
USB endpoint buffers for any given transfer? Or does the FPGA
implementation just point to the image data stored in RAM (assuming image
frame captured to some local RAM buffer) and never move the data from
image_buffer -> to_endpoint -> to_USB (essentially skipping the middle
man by just using a pointer of sorts)?

It's just a still picture (not a movie). Doesn't this seem like a somewhat
common function -- meaning wouldn't there be standard chips or
implementations that do this?

One suggestion is to use a fairly decent "microcontroller" such as
Intel Xscale (low power per MIPS) device.

USB means stack. An USB stack couldd be bought from SoftConnex, for
instance. A fitting device might be found from Transdimension.

Homann
(Yes a relation exists)
--
Magnus Homann, M.Sc. CS & E
d0asta@dtek.chalmers.se

 

[Sander Vesik]

john jakson <johnjakson@yahoo.com> wrote:

"GB" <donotspam_grantbt@jps.net> wrote in message news:<GFJ6b.2610$PE6.1782@newsread3.news.pas.earthlink.net>...
"hamilton" <hamilton@deminsional.com> wrote in message
news:3f5b561c_3@omega.dimensional.com...

What image sensor chip are you looking at ???


That's another undecided at this time, but CMOS most likely.

GB

Until recently I was under the impression that CMOS sensors were junk
and were of low resolution & quality typically 320.240 used in $0-50
cameras. I have an old Connectix webcam device thats is all green that
demonstrates that.

I believe all present digtal cameras use CMOS. That includes monsters
in the 16Mpixel range.

--
Sander

+++ Out of cheese error +++

 

[Neil Franklin]

Sander Vesik <sander@haldjas.folklore.ee> writes:

john jakson <johnjakson@yahoo.com> wrote:

Until recently I was under the impression that CMOS sensors were junk
and were of low resolution & quality typically 320.240 used in $0-50
cameras. I have an old Connectix webcam device thats is all green that
demonstrates that.

I believe all present digtal cameras use CMOS. That includes monsters
in the 16Mpixel range.

Nope. CMOS sensors are found in cheap cameras up to about 1Mpixel and
up to about $100. Above that it is all CCD sensors.


--
Neil Franklin, neil@franklin.ch.remove http://neil.franklin.ch/
Hacker, Unix Guru, El Eng HTL/BSc, Programmer, Archer, Blacksmith
- hardware runs the world, software controls the hardware
code generates the software, have you coded today?

 

[james]

On Tue, 9 Sep 2003 01:42:27 +0000 (UTC), Sander Vesik
<sander@haldjas.folklore.ee> wrote:

john jakson <johnjakson@yahoo.com> wrote:
"GB" <donotspam_grantbt@jps.net> wrote in message news:<GFJ6b.2610$PE6.1782@newsread3.news.pas.earthlink.net>...
"hamilton" <hamilton@deminsional.com> wrote in message
news:3f5b561c_3@omega.dimensional.com...

What image sensor chip are you looking at ???


That's another undecided at this time, but CMOS most likely.

GB

Until recently I was under the impression that CMOS sensors were junk
and were of low resolution & quality typically 320.240 used in $0-50
cameras. I have an old Connectix webcam device thats is all green that
demonstrates that.

I believe all present digtal cameras use CMOS. That includes monsters
in the 16Mpixel range.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

"CMOS" sensors are usually 1 megapixel or less and have most of the
timing generation integrated on board. They are less flexible than CCD
imagers which are CMOS technology.

james

 

[Jonathan Bromley]

"GB" <donotspam_grantbt@jps.net> wrote in message
news:ZMJ6b.2615$PE6.812@newsread3.news.pas.earthlink.net...

It's just a still picture (not a movie).

Which means that you may be able, after all, to do the
whole thing with a micro - if you can tolerate very
long image readout times (~2 second).

CCDs *can* output their pixels very fast - they *must*
do so for video applications, because you have to be
able to read out all 2M-ish pixels in one frame time -
but they don't *have* to. The image sensor manufacturer
probably specs some stupid-fast minimum pixel rate, but you
don't need to believe that. They simply don't want the
hassle of testing the things at low data rates. Very, very
low data rates will degrade the image because of leakage
of charge in the CCD transfer registers, but only if you
go to stupid-long readout times. So here's the deal:

Using your micro....
(1) Pulse the frame transfer clock just once, like it
says in the data sheet for the beginning of a frame.
This transfers all the accumulated photo-optical charge
from imaging cells (pixels) to transfer cells in the CCD.
The now-empty imaging cells immediately start to accumulate
photo-charge, but you can suppress this process by using the
CCD's "electronic shutter" feature - typically, this involves
pulling the anti-blooming gates to a special voltage so that
any new photo-charge is spilled off into the anti-blooming
drain.
(2) Wiggle the vertical AND horizontal transfer clocks
TOGETHER using your micro. Yes, I know the manufacturer's
data doesn't tell you to do this. All you're trying to
do here is to clean out the transfer registers as quickly
as you can. The video output will be garbage during this
process. Keep going until you have clocked the vertical
transfer at least twice as many times as there are
video lines; remember that your camera has been
exposing for a very long time before all this business
began, so there's a lot of photo-charge sloshing around
that needs to be cleared out thoroughly. Anyway, you
need only around two thousand cycles of the clock, and
you don't need to DO anything on those cycles - just
keep on clocking!
(?) It's probably a good idea to do (1) and (2) once more,
particularly if the camera has been idling in a high light
level for a long time.
(3) Stop all the clocks, to keep the camera nice and quiet.
Open the electronic shutter so that exposure can begin.
(If you prefer, you could have opened the shutter at any
time during (2). This makes the process a bit more
complicated, but quicker). Time your exposure, then:
(4) Hit the frame transfer to get your desired image
into the transfer registers. Then you can close the
electronic shutter again, so that any further illumination
does no harm.
(5) Now you need to get the image out of the camera. You
can do this at leisure, but you should follow the camera
manufacturer's *sequencing* of vertical and horizontal clocks
to the letter. Because you're in no hurry, software can do
this. Digitise the resulting outgoing video, capture it
through the micro, and you're done. As slowly as you like.

To reduce shutter latency, it's better to loop around steps
(1) and (2) continuously so that the camera is always ready
for a new exposure in no more than the time it takes to do (2).

Details will vary dramatically between cameras, but this is
do-able with any interline transfer CCD and it greatly reduces
the hardware requirements - at terrible cost of image capture
rate, of course. You'll be doing well if you can get an image
out within a second.

It's probably no use to you, but it's worth a thought if you
are hardware-averse.
--
Jonathan Bromley, Consultant

DOULOS - Developing Design Know-how
VHDL * Verilog * SystemC * Perl * Tcl/Tk * Verification * Project Services

Doulos Ltd. Church Hatch, 22 Market Place, Ringwood, Hampshire, BH24 1AW, UK
Tel: +44 (0)1425 471223 mail: jonathan.bromley@doulos.com
Fax: +44 (0)1425 471573 Web: http://www.doulos.com

The contents of this message may contain personal views which
are not the views of Doulos Ltd., unless specifically stated.

 

[james]

On Sun, 07 Sep 2003 15:03:39 GMT, "GB" <donotspam_grantbt@jps.net>
wrote:

Hi,

I'm a firmware guy pulled into a project well out of my area of
expertise. My boss wants to build (essentially) a digital camera
using an image sensor chip (1600x1200) and output it's data
"as fast as possible" using USB2.0. His initial concept, being
that I'm a firmware guy, was to use a "really fast micro."
Normally the company is involved with small 8-bitter micro
projects, so you can see I'm well out of my normal bounds.

Now this seems like a pretty big stretch to me... and I don't see
how it can be done without the speed of hardware (the image
chips all seem to have clock speeds in the tens of MHz and the
USB2 transfer rate is 480Mbps (theor.). Do aspects of this
project require an FPGA to keep the data "as fast as possible?"
If we use and FPGA for camera-to-RAM and then use a
micro for the USB2 part, what kind of fast micros can
move data at that rate?

Also, this is something that I am sure we will have to contract
out, so if you have any past experience with this, please let
me know your thoughts (and if you are available).

Thanks!

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

1600x 1200 is essentially a 2 megapixel camera!

1) First step is to determine what the camera is going to be used for!

Terrestial or astronomical or video photography

2) Pick an imager! Either Sony, TI, Kodak, or Panasonic to name a few.

3) From the Imager specs you can derive how fast the data can be
clocked out of the imager. Most imagers will transfer the image area
into a serial register one line at a time. How fast this is depends on
how fast you can clock the serial register. Transfer speeds differ
from vendor to vendor.

4) Then build the circuitry around the imager based on its ability to
transfer the full image as fast as you want and that meet your cost
goals.

Again depending on what you determine as reasonably fast will effect
the cost of the imager along with its size. Another consideration will
be the speed of the ADC. That can slow things down also. Even if you
can clock the serial register of the imager out at 20Mhz rate, if the
ADC sasmple rate is 10 MHz that is as fast as you can get the pixel
data out.

IF your imager's max clock frequency for the serial register is 20
MHz., you can clock a 1600 pixel row out in about 80 uS. Or the whole
image area out of the chip in about 100 mS. So your microC or FPGA
will have to read the ADC once every 50 nS or so during the readout
period.

There are some CPU cores as well as USB cores that can drop into an
FPGA. You can build a large FIFO or add onboard flash to store the
picture.

It is not crazy at all in fact it is quite doable. The two key items
in a digital camera is the imager and the ADC. All the rest is digital
hardware that is well suited for an ASIC or FPGA.


james

 

[Andrew Paule]

Use the micro to set up the packets in an FGPA/ASIC under isocronous
control, and stream them out from there, if you can deal with the low
data/frame rates - I used to build large format (4 x 5 and hasseblad)
camera digital inserts using big CCD's - USB (even at 12Mbs is too slow
for this stuff ( a 1Mpixel at 8 bit will require 2/3 sec + overhead to
empty one frame) - tell you boss it sounds cool, but you need 1394
(firewire) or SCSI to do this worth a DS. If you simply want to capture
one frame and empty it - consider dumping it to RAM and then out from
there. Leaving stuff on a sensor - CCD or CMOS (yes CCD's are CMOS,
both n and p type are built) results in large dark currents that make
them unusable. At 2/3 second, well depth will be a large consideration
here - electrons like to move around.

Andrew

james wrote:

On Sun, 07 Sep 2003 15:03:39 GMT, "GB" <donotspam_grantbt@jps.net
wrote:



Hi,

I'm a firmware guy pulled into a project well out of my area of
expertise. My boss wants to build (essentially) a digital camera
using an image sensor chip (1600x1200) and output it's data
"as fast as possible" using USB2.0. His initial concept, being
that I'm a firmware guy, was to use a "really fast micro."
Normally the company is involved with small 8-bitter micro
projects, so you can see I'm well out of my normal bounds.

Now this seems like a pretty big stretch to me... and I don't see
how it can be done without the speed of hardware (the image
chips all seem to have clock speeds in the tens of MHz and the
USB2 transfer rate is 480Mbps (theor.). Do aspects of this
project require an FPGA to keep the data "as fast as possible?"
If we use and FPGA for camera-to-RAM and then use a
micro for the USB2 part, what kind of fast micros can
move data at that rate?

Also, this is something that I am sure we will have to contract
out, so if you have any past experience with this, please let
me know your thoughts (and if you are available).

Thanks!



^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1600x 1200 is essentially a 2 megapixel camera!

1) First step is to determine what the camera is going to be used for!

Terrestial or astronomical or video photography

2) Pick an imager! Either Sony, TI, Kodak, or Panasonic to name a few.

3) From the Imager specs you can derive how fast the data can be
clocked out of the imager. Most imagers will transfer the image area
into a serial register one line at a time. How fast this is depends on
how fast you can clock the serial register. Transfer speeds differ
from vendor to vendor.

4) Then build the circuitry around the imager based on its ability to
transfer the full image as fast as you want and that meet your cost
goals.

Again depending on what you determine as reasonably fast will effect
the cost of the imager along with its size. Another consideration will
be the speed of the ADC. That can slow things down also. Even if you
can clock the serial register of the imager out at 20Mhz rate, if the
ADC sasmple rate is 10 MHz that is as fast as you can get the pixel
data out.

IF your imager's max clock frequency for the serial register is 20
MHz., you can clock a 1600 pixel row out in about 80 uS. Or the whole
image area out of the chip in about 100 mS. So your microC or FPGA
will have to read the ADC once every 50 nS or so during the readout
period.

There are some CPU cores as well as USB cores that can drop into an
FPGA. You can build a large FIFO or add onboard flash to store the
picture.

It is not crazy at all in fact it is quite doable. The two key items
in a digital camera is the imager and the ADC. All the rest is digital
hardware that is well suited for an ASIC or FPGA.


james

 

[james]

This is not my project. It's GB's project.

Still you can not build hardware around an unknown sensor. You have to
pick an imager and then build the hardware/firmware around it.

james


On Wed, 10 Sep 2003 20:57:32 -0500, Andrew Paule <lsboogy@qwest.net>
wrote:

Use the micro to set up the packets in an FGPA/ASIC under isocronous
control, and stream them out from there, if you can deal with the low
data/frame rates - I used to build large format (4 x 5 and hasseblad)
camera digital inserts using big CCD's - USB (even at 12Mbs is too slow
for this stuff ( a 1Mpixel at 8 bit will require 2/3 sec + overhead to
empty one frame) - tell you boss it sounds cool, but you need 1394
(firewire) or SCSI to do this worth a DS. If you simply want to capture
one frame and empty it - consider dumping it to RAM and then out from
there. Leaving stuff on a sensor - CCD or CMOS (yes CCD's are CMOS,
both n and p type are built) results in large dark currents that make
them unusable. At 2/3 second, well depth will be a large consideration
here - electrons like to move around.

Andrew

james wrote:

On Sun, 07 Sep 2003 15:03:39 GMT, "GB" <donotspam_grantbt@jps.net
wrote:



Hi,

I'm a firmware guy pulled into a project well out of my area of
expertise. My boss wants to build (essentially) a digital camera
using an image sensor chip (1600x1200) and output it's data
"as fast as possible" using USB2.0. His initial concept, being
that I'm a firmware guy, was to use a "really fast micro."
Normally the company is involved with small 8-bitter micro
projects, so you can see I'm well out of my normal bounds.

Now this seems like a pretty big stretch to me... and I don't see
how it can be done without the speed of hardware (the image
chips all seem to have clock speeds in the tens of MHz and the
USB2 transfer rate is 480Mbps (theor.). Do aspects of this
project require an FPGA to keep the data "as fast as possible?"
If we use and FPGA for camera-to-RAM and then use a
micro for the USB2 part, what kind of fast micros can
move data at that rate?

Also, this is something that I am sure we will have to contract
out, so if you have any past experience with this, please let
me know your thoughts (and if you are available).

Thanks!



^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1600x 1200 is essentially a 2 megapixel camera!

1) First step is to determine what the camera is going to be used for!

Terrestial or astronomical or video photography

2) Pick an imager! Either Sony, TI, Kodak, or Panasonic to name a few.

3) From the Imager specs you can derive how fast the data can be
clocked out of the imager. Most imagers will transfer the image area
into a serial register one line at a time. How fast this is depends on
how fast you can clock the serial register. Transfer speeds differ
from vendor to vendor.

4) Then build the circuitry around the imager based on its ability to
transfer the full image as fast as you want and that meet your cost
goals.

Again depending on what you determine as reasonably fast will effect
the cost of the imager along with its size. Another consideration will
be the speed of the ADC. That can slow things down also. Even if you
can clock the serial register of the imager out at 20Mhz rate, if the
ADC sasmple rate is 10 MHz that is as fast as you can get the pixel
data out.

IF your imager's max clock frequency for the serial register is 20
MHz., you can clock a 1600 pixel row out in about 80 uS. Or the whole
image area out of the chip in about 100 mS. So your microC or FPGA
will have to read the ADC once every 50 nS or so during the readout
period.

There are some CPU cores as well as USB cores that can drop into an
FPGA. You can build a large FIFO or add onboard flash to store the
picture.

It is not crazy at all in fact it is quite doable. The two key items
in a digital camera is the imager and the ADC. All the rest is digital
hardware that is well suited for an ASIC or FPGA.


james

 

[Andrew Paule]

Sorry - no offense meant.

Andrew

james wrote:

This is not my project. It's GB's project.

Still you can not build hardware around an unknown sensor. You have to
pick an imager and then build the hardware/firmware around it.

james


On Wed, 10 Sep 2003 20:57:32 -0500, Andrew Paule <lsboogy@qwest.net
wrote:



Use the micro to set up the packets in an FGPA/ASIC under isocronous
control, and stream them out from there, if you can deal with the low
data/frame rates - I used to build large format (4 x 5 and hasseblad)
camera digital inserts using big CCD's - USB (even at 12Mbs is too slow
for this stuff ( a 1Mpixel at 8 bit will require 2/3 sec + overhead to
empty one frame) - tell you boss it sounds cool, but you need 1394
(firewire) or SCSI to do this worth a DS. If you simply want to capture
one frame and empty it - consider dumping it to RAM and then out from
there. Leaving stuff on a sensor - CCD or CMOS (yes CCD's are CMOS,
both n and p type are built) results in large dark currents that make
them unusable. At 2/3 second, well depth will be a large consideration
here - electrons like to move around.

Andrew

james wrote:



On Sun, 07 Sep 2003 15:03:39 GMT, "GB" <donotspam_grantbt@jps.net
wrote:





Hi,

I'm a firmware guy pulled into a project well out of my area of
expertise. My boss wants to build (essentially) a digital camera
using an image sensor chip (1600x1200) and output it's data
"as fast as possible" using USB2.0. His initial concept, being
that I'm a firmware guy, was to use a "really fast micro."
Normally the company is involved with small 8-bitter micro
projects, so you can see I'm well out of my normal bounds.

Now this seems like a pretty big stretch to me... and I don't see
how it can be done without the speed of hardware (the image
chips all seem to have clock speeds in the tens of MHz and the
USB2 transfer rate is 480Mbps (theor.). Do aspects of this
project require an FPGA to keep the data "as fast as possible?"
If we use and FPGA for camera-to-RAM and then use a
micro for the USB2 part, what kind of fast micros can
move data at that rate?

Also, this is something that I am sure we will have to contract
out, so if you have any past experience with this, please let
me know your thoughts (and if you are available).

Thanks!





^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1600x 1200 is essentially a 2 megapixel camera!

1) First step is to determine what the camera is going to be used for!

Terrestial or astronomical or video photography

2) Pick an imager! Either Sony, TI, Kodak, or Panasonic to name a few.

3) From the Imager specs you can derive how fast the data can be
clocked out of the imager. Most imagers will transfer the image area
into a serial register one line at a time. How fast this is depends on
how fast you can clock the serial register. Transfer speeds differ


from vendor to vendor.


4) Then build the circuitry around the imager based on its ability to
transfer the full image as fast as you want and that meet your cost
goals.

Again depending on what you determine as reasonably fast will effect
the cost of the imager along with its size. Another consideration will
be the speed of the ADC. That can slow things down also. Even if you
can clock the serial register of the imager out at 20Mhz rate, if the
ADC sasmple rate is 10 MHz that is as fast as you can get the pixel
data out.

IF your imager's max clock frequency for the serial register is 20
MHz., you can clock a 1600 pixel row out in about 80 uS. Or the whole
image area out of the chip in about 100 mS. So your microC or FPGA
will have to read the ADC once every 50 nS or so during the readout
period.

There are some CPU cores as well as USB cores that can drop into an
FPGA. You can build a large FIFO or add onboard flash to store the
picture.

It is not crazy at all in fact it is quite doable. The two key items
in a digital camera is the imager and the ADC. All the rest is digital
hardware that is well suited for an ASIC or FPGA.


james

 

[Neil Franklin]

Andrew Paule <lsboogy@qwest.net> writes:

USB (even at 12Mbs is too slow
for this stuff ( a 1Mpixel at 8 bit will require 2/3 sec + overhead to
empty one frame) - tell you boss it sounds cool, but you need 1394
(firewire) or SCSI to do this worth a DS.

According to the Subject: line he is intending to use USB2.

That is 480MBit/s, according to http://www.usb.org/faq/ans2#q1 .

Should be fast enough.


--
Neil Franklin, neil@franklin.ch.remove http://neil.franklin.ch/
Hacker, Unix Guru, El Eng HTL/BSc, Programmer, Archer, Blacksmith
- hardware runs the world, software controls the hardware
code generates the software, have you coded today?