Questions tagged [accuracy]
The accuracy tag has no summary.
148 questions
0 votes
6 answers
122 views
High-accuracy DC current and voltage measurement on PCB
My application requires a high accuracy reading for DC voltage and current. DC voltage max is about 60 V and DC current max is about 140 A. Currently I am using a 1 mΩ shunt(CSS2H-3920R-1L00F), for ...
- 493
1 vote
2 answers
96 views
Are LDO line and load regulation specs useless when the tolerance is so loose?
I'm looking at this ST1L05 voltage regulator for the purpose of an uncertainty/error analysis of a downstream ADC. I'm struck by how loose the output voltage tolerance range is, at least for the 3V3 ...
- 240
3 votes
4 answers
222 views
Measuring wide voltage range via voltage divider "accurately"
I'd like to measure the voltage range 3V .. 30V via a voltage divider. When I use a ratio of 100k to 10k for example and use 1% resistors (taking into account drift due to aging -> 2% or even more) ...
- 674
4 votes
7 answers
2k views
Precise Electronic Measurements
I've been looking into various electronic components and noticed rampant 5%, 10%, and even 20% tolerances and accuracies not only in basic components such as capacitors, inductors, and resistors, but ...
- 267
0 votes
1 answer
387 views
What is the best and simplest way to generate multiple accurate reference voltages?
I need several reference DC voltages to use them to calibrate hundreds of circuit boards manufactured in our facility. The voltages to be calibrated are 5V 10V 100V 200V 500V The accuracy required ...
- 129
1 vote
3 answers
661 views
Easiest way to get highly-accurate frequency reference?
I just want to check the calibration of my frequency counter, so anything constant under 250 MHz would work(?) Mains are synced to atomic clocks, to maintain wall clock accuracy, but only ...
- 29.3k
1 vote
3 answers
476 views
Shunt's thermal EMF (Seebeck effect) vs Kelvin connection?
While reading this Vishay document about shunts, I saw that temperature mismatch between the shunt's terminals can introduce a "parasitic" voltage drop which will in turn affects the current ...
- 916
2 votes
1 answer
109 views
Which INL is better for this ADC?
I am using the following ADC. I want to know given the figures below, which INL is better? How do I interpret the x-axis (% of FS)? What is my full-scale (FS) value? My Vin ranges from 10mV to 50mV ...
- 2,980