When I first start the PV2 hacking project, I was wandering if it is possible to use the CMOS sensor on the PV2 for some other projects. Although it took a lot longer than I expected to reach my first milestone, which is to get some data out of the camera's sensor. Here I try to put together as much of my finding as I could. This hack is quite complicated and understanding of digital circuit design will be required to understand the process of extracting the data. I'll try my best to describe all the details.
Some basic facts about the camera. The camera has a CMOS sensor that put out 8 bit data. The sensor has a resolution of 1300 x 1024.
Let's start by looking at the camera after the case has been removed:
Three parts can be seen that made up the entire camera. The board on the top is the strobe light section. We can remove this completely. Please refer to my other PV2 page regarding how the stobe light section works. The small LCD display can be seen in the middle of the main circuit board. The CMOS sensor is mounted on the other side of the main circuit board.
After a bit of visual observation of the circuit board, it was fairly obvious the test pads beside the LCD display are connected directly to the CMOS sensor chip. This is a very nice feature, as it allows the signal on the CMOS sensor to be probed without remove the lens assembly, wich is required to focous a good image on the sensor. Here is the signal extracted. Notice the LCD display has been removed to allow access to all the probe pads.
Description of the pins:
D0-D7 - Data lines Vsync - This is actually the horizontal sync Bclk - This is the bit clock at 48MHz V - Data valid cmd - Command. This pin toggles when command is issued into the CMOS sensor 1.93MHz - Like what it says, it is 1.93MHz, I do not know what it is forIf you are interested in how it is connected to the actual CMOS chip, here is the detail:
* - Ping 1 marker here Vss -- | | -- Analog signal Sync -- | | -- Analog signal D7 -- | | -- Analog signal D6 -- | | -- Analog signal D5 -- | | -- Analog signal D4 -- | | -- Analog signal D3 -- | | -- Analog signal D2 -- | | -- Analog signal D1 -- | | -- 1.93 MHz D0 -- | | -- GND GND -- | | -- CMD Bclk -- | | -- VPlaying with the scope, I can see that the CMOS sensor is being driven continuously. Here is some of the timing relationship I have extracted from the signals.
It seems the CMOS sensor has been put on continuous capturing mode. This is good as I can create a passive sniffer to extract the data from the CMOS sensor chip without actually driving the chip. No cutting of the PCB trace is required, and I would not have to figure out how to configure the sensor chip.
From the timing diagram, it is also possible to extract some basic information regarding how the CMOS chip is being driven. In the second diagram, we can see a single frame takes 24.6mS to complete. Each horzontal line is about 21.4 + 28.6 = 50uS to complete. Simple division shows there is roughly 490 lines of data in a single frame. Also, using the bit clock speed, which is 48MHz, we can calculate number of pixels in a single horizontal line. 21.4uS / (1 / 48MHz) give roughly 1300 pixels per line.
It is time to wire up the sensor, using blue wire to wire the signal to a ribbon cable, which is superglued to the PCB.
The other side of the ribbon cable is connected to my trusty FPGA development board from Digilent.
The goal here is to extract the data from the CMOS sensor. However, the chip is outputing at quite a high speed. Since the bit clock is running at 48MHz, the data rate is at 48M byte per second. There is no simple way to pipe this data into the PC directly.
This is the time for some FPGA programming. The idea is to use the onchip BRAM as a small FIFO to hold a single line of the data. This data is transfered into the larger PSRAM. Once a single frame of picture is captured, the process stops. From the PC side, the data is extracted from the PSRAM through the USB port. Here is a summary of the entire system:
This is the final image extracted from the sensor. You can see the image has been repeated 3 times. From the image, I can extract that only 480 lines has been captured. This indicates the camera is most likely running in web cam mode. The output resolution of the webcam mode should be 640 x 480. However, the sensor is capture the entire 1300 horizontal pixels. It seems the camera can be made into a nice wide-screen webcam.
One issue that the current FPGA firmware has is it only capture 928 horizontal lines instead of the full 1300 lines. The reason for this is because my PSRAM controller can not handle burst write mode yet. As a result, the write time for a single 16 bit word takes 80ns (WR low) + 20ns (CS high between each write). There is not sufficient time between each line of data to capture everything. As a result, some data is dropped on purpose.
Another outstanding task is the Hsync signal is not used. As a result, the horizontal position of the picture is not fixed. This is fine for a proof of concept project. It would be nice if I can find some time to refine the project.
People with keen eyes probably notice the pattern on the extracted picture. The checker pattern is caused by the colour filter on top of the CMOS sensor. The filter has a typical Bayer filter pattern. Thus, with a bit of programming, it is possible to extract colour picture from the sensor.