A fast analog front-end processor for digital imaging systems

A FAST ANALOG FRONT-END PROCESSOR FOR DIGITAL IMAGING SYSTEMS WITH HIGHER SENSOR RESOLUTIONS AVAILABLE, THE SPEED AND DYNAMIC RANGE REQUIREMENTS FOR IMAGE PROCESSORS IN DIGITAL IMAGING SYSTEMS ARE MORE DEMANDING. A 12-BIT, 50 MPIXELS/S DIGITAL IMAGE ACQUISITION SYSTEM BALANCES POWER AND PERFORMANCE. THE ANALOG PROCESSOR’S TOTAL POWER DISSIPATION IS ONLY 150 MW AT FULL SPEED— AN ENVIABLE QUALITY FOR THE PORTABLE MARKET. ALTHOUGH THE UNDERLYING TECHNIQUE REQUIRES SPECIAL ANALOG CIRCUITRY TO HANDLE FAST GAIN CHANGES, IT ACHIEVES A FAR WIDER DYNAMIC RANGE. According to Photo Marketing Asso- The increased sensor resolution leads to a ciation International (http://www.pmai.org), smaller pixel size and a lower available signal- high-end consumer and professional applica- to-noise ratio (SNR). As Figure 2 shows, a low- tions in a wide range of digital image acquisi- resolution image (300 Kpixels, on the left) looks tion systems, such as digital video camcorders, more pleasant than a high-resolution image (1.3 digital still cameras, PC video teleconferencing, Mpixels, on the right) if the dynamic range is Ion E. Opris digital copiers, and infrared image digitizers, reduced to less than 50 dB. It is a surprising phe- require ever better image quality (http://www. nomena, but almost all consumer-grade cam- Independent consultants pmai.org/studies/00cps-us.htm). With higher- eras that are 1 Mpixel or higher have this resolution sensors available, the pressure is on problem. High image quality requires a global the electronic circuitry to handle a large num- SNR better than 60 dB. This SNR is actually Seiichiro Watanabe ber of pixels at reasonable scan speeds and power close to the sensor capability, which requires bet- consumption. For example, digital still cameras ter electronic-circuitry performance. Figure 3 NuCore Technology Inc. require a minimum of 1 Mpixel/image, and dig- shows that actual image quality differs in the ital video needs 30 frame/s. Another profes- dark area between the NC1250 12-bit, 62-dB sional application requires the acquisition of a SNR analog front end (AFE) and the conven- 4-Mpixel high-quality digital picture at 12 tional 10-bit, less-than-55-dB SNR AFE. frames/s. The current trends in various appli- A related issue to the dynamic range is the cations, shown in Figure 1, will require pro- white balance. Current single-sensor AFE sys- cessing speeds of about 50 Mpixels/s. tems do not have an analog white-balance capa- 48 0272-1732/01/$10.00  2001 IEEE  7 6 10 bit 12 bit 5 DSC 4 Mpixel 3 DVC High-definition TV 2 1 10 Mpixel/s 50 Mpixel/s 0 0 10 20 30 40 50 60 70 Frame/s Figure 2. Dynamic range issue with high-resolution images. Figure 1. Current trend requirements for digital image acquisi- tion systems. Figure 3. A 12-bit analog-to-digital converter and 62-dB SNR image(NC1250) compared with a 10-bit analog-to-digital con- Figure 4. Color noise in half-tone areas without analog white verter and 55-dB SNR image. balance. Green A/D output granularity Blue Red Light intensity Figure 5. Effect of analog white balance (using the same Sony ICX205AK/CCD device): Figure 6. Color noise due to different quantization errors. NC1250 using NDX technology, analog white balance (top) and Sony CCD reference system with digital white balance (bottom). gradation. NC1250 processed the upper exam- ple with analog white balance and processed the lower example with digital white balance. bility. Figure 4 shows a color noise in half-tone Note that some color bands appear in the dig- areas. Figure 5 shows the apple-to-apple com- ital white-balance example. Different quanti- parison between analog white balance and dig- zation errors on each independent color cause ital white balance in the case of gray-scale this color noise, as shown in Figure 6. MARCH–APRIL 2001 49  DIGITAL IMAGE SYSTEM NDX CF memory CCD Analog image Digital image Compression Media Hard disk A/D Ethernet sensor processing processing and formatting interface USB TV/HDTV NC1250 SiP1250 Figure 7. Digital image-acquisition system. Another necessary requirement for the AFE whereas all the digital circuitry, including the is small power dissipation. Especially for digital-signal processing, performs at 1.8 V portable applications, smaller power dissipa- for power reduction. These power-supply tion increases battery life. constraints require different CMOS tech- This article discusses the implementation of nologies used in the AFE, which we imple- the digital image acquisition system, with an mented in 0.35 µm, versus the digital post emphasis on the AFE sensor processor. Our processor implemented in 0.18 µm. goal was to develop a complete digital acqui- The technical challenges in the AFE are to sition system from sensor to storage, capable maintain a 12-bit accuracy along the entire of acquiring, enhancing, compressing, dis- analog processing chain, achieve a color-by- playing, and storing high-quality digital images color gain control at pixel speed, and keep the continuously at 50 Mpixels/s with a minimum power dissipation low for throughputs in overall SNR of 60 dB and minimum power excess of 50 Mpixels/s. The digital post dissipation. A proprietary NDXTM technique processor also keeps a 12-bit accuracy, which enables white balance in the analog domain, needs more bits inside the process and per- equalizing each color dynamic range before the forms 20 corrections on each pixel, including analog-to-digital conversion. bad-pixel information replacement, color cor- A unique feature of the AFE is the hori- rection, gamma correction, and motion JPEG zontal decimation performed in the analog compression. This processor can interface domain. Decimation is a technique that trans- with various storage and display media, such forms an image with many pixels into an iden- as compact flash memory, hard disk, Ether- tical image with fewer pixels. This technique net, USB, and TV or high-definition TV. All is useful when viewing the image on lower- the analog and digital signal processing occurs resolution displays or in preview modes. By in pipelines with a clock period of 20 ns. decimating the image in the analog domain, the rest of the processing chain, including the System description analog-to-digital converter (ADC) and the The AFE, as shown in Figure 8, includes a digital post processor, can process the infor- high-speed correlated double sampler (CDS), mation at lower clock rates, achieving impor- an 8-bit digitally programmable gain amplifi- tant power savings. er (PGA), a 12-bit ADC, and digital black- level auto-calibration circuitry. System architecture A CDS is commonly used for a CCD device Figure 7 shows the digital image acquisi- for canceling low-frequency noises, which tion system. It includes a single CCD image mainly consist of 1/f noise and reset noise. sensor and associated optics, an analog image- CDS is based on a differential amplifier and a processing unit, an ADC, a digital image pro- time-shifted pair of sample hold circuits. cessing unit, compression and formatting The CDS subtracts the CCD output signal circuitry, and a media interface. We optimized black level from the video level. The differen- the system partitioning to achieve the best per- tial CDS input stage rejects common-mode formance in the processing chain with mini- signal noise and power-supply noise. The PGA mum power dissipation. is digitally controlled with 8-bit resolution on The analog circuitry required a supply volt- a logarithmic scale, resulting in a gain adjust age of 3.3 V for improved dynamic range, range of 32 dB with 0.125 dB per least-signif- 50 IEEE MICRO  OB_CLP RLAT_B DLAT_B VREFN VREFP RESET DTEST X_CLP SWAP HS_B VREF VS_B VMID Reference Clock generator Test generator ADC_CLK NDX SOF CCD_IN1 Data CDS Programmable 12-bit ADC VALID_PIX interface CCD_IN2 gain amplifier INT DATA<11:0> Power SDATA management Serial Black level SCLK timing control interface correction SDEN Figure 8. Analog front end (AFE) block diagram. icant bit (LSB) of the gain code. The timing tracts the reset from the data level and ampli- and gain control are compatible with a wide fies the difference. The subtraction effective- variety of CCD and CMOS image sensors. ly cancels the CCD output amplifier, A simple three-wire serial interface controls low-frequency noise, and offset. the PGA, black-level auto-calibration, and The CDS block also converts the inherent- timing circuitry. Internal registers controlled ly single-ended input from the CCD to a dif- by the serial interface contain information ferential output signal to improve the about the color pattern and sensor size (num- signal-processing performance of the down- ber of pixels) and optimize the power dissipa- stream blocks in the AFE. A 10-bit offset dig- tion for pixel speed. ital-to-analog converter (DAC) helps convert the single-ended signal to a differential output Operation and circuit implementation signal and does the analog-black calibration. Our application consists of five major func- We implemented all the analog processing tion blocks. We explain how each function circuits with switched-capacitor circuits. The works and the key ideas behind each imple- CDS block actually contains two sets of capac- mentation. itor arrays to perform a ping-pong signal-pro- cessing operation, shown in Figure 9 (next Correlated double sampler page). This ping-pong operation reduces The CDS block performs the correlated power dissipation for the same noise and speed double sampling to remove low-frequency performance because the same amplifier is used noise and offset from the CCD output signal. for both arrays. This ping-pong scheme cut CCD_IN1 is the actual CCD output signal, the speed of each in half. Each amplifier can and CCD_IN2 is a pseudo-differential input save the power for the load capacitor (the sum capacitively tied to either the positive CCD total of capacitors connected) and reduce the supply or grounded to improve power-supply static current for the speed. rejection. After sampling the reset and data The capacitor array includes the correlated levels for a given pixel, the CDS block sub- double-sampling circuitry and a 10-bit offset MARCH–APRIL 2001 51  DIGITAL IMAGE SYSTEM color and then averaging them to reduce the spatial- Multi switched-cap banks frequency content of the pixel data. This averaging reduces the output data rate from the Vin CDS block, enabling all sig- PGA1 PGA1 PGA2 PGA3 PGA4 PGA5 nal-processing blocks that fol- low the CDS block to operate at lower clock frequencies and thus dissipate less power. The 10-bit D/A CDS block can perform converter vga(0)−vga(3) vga(4)−vga(7) either 2x or 3x decimation, reducing the output data rate Figure 9. The NDX architecture for the correlated double sampler (CDS) and the programma- by a factor of 2 or 3 (relative ble gain amplifier. to the input data rate) by averaging two or three suc- cessive pixels of the same Vrefp Vcom Vrefn color. In the no-decimation mode, the input and output data rates are the same. Reference buffers Vinp Vinn Programmable gain amplifier The block diagram in Fig- ure 9 also shows the PGA bias clk Conversation core (17 stages) implemented with five gain power control stages, each with approxi- mately equal maximum gain 18 (data) to provide maximum speed at 12 (cal) minimum power dissipation Calibration/digital for the desired noise and dis- 12 tortion. The PGA takes the Correction Dout Dout<12:1> CDS’s differential output and amplifies it under the control Figure 10. Analog-to-digital converter (ADC) block diagram. of an 8-bit programmable word from 0 to 32 dB in 0.125-dB steps. DAC. Thus, a given capacitor array samples The PGA can have four different gain set- the current pixel while the other capacitor tings, to accommodate four colors, and these array holds the information from the previous gains can be switched at the pixel rate. color pixel. Analog black-level calibration is performed for each color separately. Each color Analog-to-digital converter has its own offset register, which allows four The ADC module performs the analog-to- different offsets and black-level corrections for digital conversion. The module includes a con- different colors. Because each color can have version core, a calibration and digital different gain settings, independent offsets and correction circuit, internal reference buffers, a black level are necessary for each color. and support and bias circuitry, as shown in Fig- ure 10. The conversion core and digital cor- Horizontal decimation rection circuitry are implemented using a dual To reduce the power dissipation even fur- pipeline. The two pipelines share the opera- ther, the CDS block can perform horizontal tional amplifiers for lower power dissipation. decimation under the control of a program- The conversion core, shown in Figure 11, has mable register. Horizontal decimation 17 conversion stages. The ADC incorporates involves sampling successive pixels of the same a digital calibration and correction mechanism 52 IEEE MICRO  Dual switched-cap banks D (18) D (17) D (16) D (15) D (14) D (13) D (12) D (11) D (10) D (9) D (8) D (7) D (6) D (5) D (3) D (3) D (2) D (1) Figure 11. ADC conversion core. based on the digital calibration of all the tran- IDLE sitions in the transfer characteristic. The dig- ital calibration algorithm is invoked at power up, and the firmware can also request it. The DIG_BLK_CAL/ ANA_BLK_CAL/ BLK_CAL/ A/D calibration should be exercised for any change in clock speed and power setting, including changes in the decimation mode. Black-level calibration DIGITAL_CAL: ANALOG_CAL: BLACK_CAL: The black-level calibration breaks down into the three parts shown in Figure 12: The coarse- Figure 12. Black-level calibration. analog black-level calibration (BLACK_CAL), the fine-analog black-level calibration (ANA- LOG_CAL), and the digital black-level cali- ues of the offset registers for eight iterations bration (DIGITAL_CAL). to bring the ADC outputs for the black pix- The coarse-analog black-level calibration els within certain bounds. The offsets will be operation determines the values of the analog adjusted in each iteration by only 1 LSB, so offset registers for all four colors. Because there each offset may be adjusted by a maximum are 10 bits for each analog offset register, fin- of 8 LSBs. If necessary, the microprocessor ishing this operation will take 10 iterations. can simply pre-program all the offset regis- At the beginning, all offsets for each color are ters and then start the fine-analog black cal- set to zero. An external signal (OB_CLAMP) ibration without first performing a coarse acts as the black-pixel indicator for the black- black-level calibration. level calibration, and two pixels from each row Finally, the digital black-level calibration are used for the black-level calibration. Com- removes the remaining black-level offset in pleting this operation requires 20 rows. The the analog circuits by averaging several black ADC output data for the selected black pix- pixels. The number of black pixels used in els are compared against an ideal zero value. averaging is register programmable. The dig- The successive approximation operation ital black-level offset will be subtracted from repeats until all the offset bits are properly set. nonblack pixels to remove the residue black The purpose of the fine-analog black-level offset. For continuous digital image acquisi- calibration operation is to fine-tune the val- tion, such as video, the digital black-level cal- MARCH–APRIL 2001 53  DIGITAL IMAGE SYSTEM separate optimizations of analog and digital Glossary signal processing blocks. The AFE described ADC Analog to digital converter. is, to our knowledge, the fastest digital imag- AFE Analog front end processor. ing acquisition system on the digital-camera Black cal Black-level calibration; the calibration needed to cancel market, with over 62 dB of dynamic range. the optical black value’s deviation from true black level. The performance was enhanced by various CCD Charge coupled device image sensor. analog and digital calibration techniques, CDS Correlated double sampler. including digital calibration and correction of FEP Front-end processor. both analog offsets and ADC nonlinearities. INL/DNL Indicator for monotonicity and linearity of ADC. We also achieved lower power dissipation by LSB Least significant bit. intensively using analog processing blocks Mpixel Mega pixel (1,000,000 pixel per picture). based on switched-capacitor techniques and MSB Most significant bit. dual-pipeline architectures. MICRO Optical black level CCD image sensors that have optical black area physically References covered by black paint or metal for calibration. 1. I. Opris et al., “A 12-Bit, 50 Mpixel/s Analog PGA Programmable gain amplifier. Front End Processor for Digital Imaging SNR Signal to noise ratio. Systems,” Hot Chips 12 Conference Record, Palo Alto, Aug. 2000; http://www.hotchips.org/index12.html. ibration occurs in background, during the 2. I. Opris, L. Lewicki, and B.C. Wong, “A inactive part of the scan. Single-Ended Input 12-Bit 20MS/s A/D Converter,” IEEE J. Solid-State Circuits, vol. Experimental results SC-33, no. 12, Dec. 1998, pp. 1898-1903. We implemented the AFE in a 0.35-µm CMOS technology, as shown in Figure 11. The functionality and performance have been Acknowledgment verified at over 50 Mpixels/s. The global SNR We thank Jonathan Kleks, Yasunori achieved was better than 63.5 dB (not includ- Noguchi, James Castillo, Shiyin Siou, Murty ing the sensor), which translates to about 130 Bhavana, Youichi Nakasone, and Shingo µV rms input-referred noise. Separate evalu- Kokuda for their help in writing this article. ations of the entire system based on residual noise on gray images indicate better than 62 Ion E. Opris is an independent consultant in dB SNR (referred to as a full-scale sine wave the areas of data acquisition, clock recovery, input), equivalent to better than 10 effective analog filters, and data communication cir- number of bits (ENOB). cuits, including RF. Opris has an Engineer The global differential nonlinearity (DNL) degree from Bucharest Polytechnic Institute was below 0.4 LSB (at the 12-bit level), and the and a MS and PhD in electrical engineering integral nonlinearity (INL) was less than 1 LSB. from Stanford University. He has published Figure 12 shows a sample image acquired with over 20 technical papers, holds 18 patents, the current system at 3.3 Mpixels. and has six pending applications in various The total power dissipation for the AFE was areas of circuit design, including data conver- between 150 mW at full speed (50 Mpixels/s) sion architectures. and around 75 mW at 30 Mpixels/s with no decimation. To change the power dissipation, Seiichiro Watanabe is principal founder of select the power-mode table setting by chang- NuCore Technology Inc. in Santa Clara, ing OP-amp bias. Calif. His research is in the area of develop- ing very high speed over 12bits-granularity. A careful partition of the digital image acquisition system between an analog front end and a digital post processor allowed Watanabe has an Engineer degree from Keio University. He was a member of the MITI industrial structure council. 54 IEEE MICRO