adjusting for RTC drift

edit:
this now has its own examples and library posted on github
https://github.com/ag88/stm32duino_rtcadj
more details here
viewtopic.php?f=18&t=4365&start=10#p51483

i happened to be using those 20 for a dollar 32k crystals
https://www.ebay.com/sch/i.html?_from=R … 8&_sacat=0
and i’m having/using one that drifts like a grandfather’s clock, er em drifts 30 secs a day
actually i think part of the reason for the drift is that i simply soldered a crystal at PC14 and PC15 pins on my maple mini – no caps

note that with the better crystals, stm32f1x has a register for you to slow down the clock up to 314 seconds per month (30 days)
https://www.st.com/content/ccc/resource … 167326.pdf
that is pretty good calibration for the *good* crystals

but if you are using *cheap* crystals like me and you don’t need parts-per-million precision
and
you are powering VBAT (maple mini has a pin for that, i’m not too sure if blue pill has it) on a coin cell

here is how i ‘adjust’ that drift of my RTC
// this is the number of seconds for the stm32 rtc to drift 1 secs (faster) from an accurate time source
#define DRIFT_DUR 3335

void setup() {
time_t last;

bkp_init();
last = getbktime();
if(last == 0) {
setbktime(rt.getTime());
} else {
/* adjust for time drift */
time_t now = rt.getTime();
if(now - last > 24 * 60 * 60) {
int dur = now - last;
int adj = dur / DRIFT_DUR; // number of sec to subtract
now -= adj; // subtracting as the clock runs fast, change this if you are 'speeding up' the rtc
rt.setTime(now);
setbktime(now);
}
}
}

void mysettime(time_t time) {
rt.setTime(time);
setbktime(time);
}

void setbktime(time_t time) {
bkp_enable_writes();
bkp_write(1, time & 0xffff);
bkp_write(2, time >> 16);
bkp_disable_writes();
}

time_t getbktime() {
time_t time;
time = bkp_read(1);
time |= bkp_read(2) << 16;
return time;
}

+1 I have these cheap crystal too!

-1

Cheap xtals are never a good buy I.M.O.
I would rather have 10 stable crystals than 100 sucky ones!

Ray

Interesting code

I wonder if it could be added to the examples in the LibMaple core (so it does not get lost)

lol

@ray
i’d agree that better crystals should make a difference, the other reason my xtal drift so much is that i simply soldered the 32k crystal to the maple mini pins no caps

i stumbled into an interesting article about the load capacitor
https://www.researchgate.net/profile/Ha … tn-021.pdf
accordingly without a load capacitor the crystal would simply oscillate at its natural resonance frequency and using a load capacitor would ‘pull’ that resonant frequency lower. using a large load capacitor may have a benefit that the frequency less ‘pullable’
however, i read some articles including stm’s own ‘getting started’ guide
https://www.st.com/content/ccc/resource … 164185.pdf
2.2. LSE OSC clock p13 which suggest
To avoid exceeding the maximum value of C L1 and C L2 (15 pF) it is strongly recommended to use a resonator with a load capacitance CL ≤ 7 pF. Never use a resonator with a load capacitance of 12.5 pF

and the reason could be that a large load capacitor may lead to the crystal failing to oscillate

it is sort of a blessing in disguise that i soldered the 32k crystal on the maple mini without caps and it oscillates at its natural frequency in its featherweight mode, it always oscillates never fails

@roger
i’d leave it to you as this may be simply an example of using the backup registers. i’m actually still testing out my codes
this would probably help in cases where the drift is large, for a smaller drift stm actually has another backup register BKP_RTCCR which could be set to slow down the RTC
https://www.st.com/content/ccc/resource … 167326.pdf
BKP_RTCCR allows one to slow down the RTC to the tune of up to 314 secs per 30 days (month) and is a ‘once off’ setting
which is needed every new coin cell replacement as BKP_RTCCR is a backup register running on VBAT

just an update, the codes seem to work rather well, however, in determining DRIFT_DUR
one way is to monitor the drift over a period of time e.g.

• sync the RTC to accurate time at the start of monitoring << i did this manually, take note of this time as the monitoring start date/time
• elapsed_time = monitor end date/time – monitor start date/time << in secs
• drift <- the total secs of drift measured against accurate time << i measured this manually, as the RTC is synced at the start of monitoring, after the elapsed time the RTC would have drifted away from accurate clock time, hence i observe the RTC time and minus the accurate clock time manually, that is the drift in secs
• DRIFT_DUR = elapsed_time / drift

the trouble is that if the monitoring duration is short, the drift in secs is likely to be low, and as i’m measuring manually by checking against pc time synced against ntp, the granuality i can measure is +/- 1 secs may possibly be more, and if the drift in secs is say 1 secs, between 1-2 secs is a 50% difference on DRIFT_DUR !
hence, one would need to monitor over a longer time to cumulate a larger drift if you are measuring manually, other automated ways may require programming hence i made do with doing so manually

i also noted that the drift varies in a rather large range. In my case i’m getting between 21 secs per day to 25 secs per day of drift that’s quite a variation.
my guess is that it may be partly be due to temperatures and more likely as i alternate between letting it run off the coin cell and connecting to power, the voltage /power differences alone could make a difference

but given the variation, this adjustment would at best move it to an average lower offset rather than having a large systematic offset.
i’d think using the proper caps etc would make a difference. but if one is willing to spend a little, a good crystal may be all that’s needed to keep the RTC on the dot no adjustments needed

“A good crystal…”

Finally, you arrive at the truth.

Ray

[mrburnette – Fri Nov 30, 2018 3:14 pm] –
“A good crystal…”

Finally, you arrive at the truth.

Ray

Actually, funny thing. On the Murata CMWX1ZZABZ you have a 32768Hz crystal and a 32MHz TCXO. I added code in ArduinoCore-stm32l0 to adjust the drift/period of the RTC using the 32MHz TCXO (which has 2ppm as opposed to the 32768Hz crystal, which has 20ppm). Made all the difference in the world.

Also fooled around with using the PPS of a GNSS to crosscorrect the alignment of the RTC. Somewhat less successful were the attempts to adjust the period of the RTC via GNSS … Turns out the TCXO correction was better in practice …

[GrumpyOldPizza – Tue Dec 04, 2018 3:23 pm] –
…
Turns out the TCXO correction was better in practice …

The first time I saw something like that was many years back with V-USB:
https://forums.obdev.at/viewtopic.php?t=1836

The above was 10 years ago… and I suspect the original implementation was earlier.

Ray

hello. I have too bad crystal and looking for solution, how to use it. Found this method.
I not understand why #define DRIFT_DUR 3335
if result of function gettime in seconds, why every day we take 3335 seconds, not 30 seconds?
Here result of now and last variables in console output
Now: 1544354802
Last: 1544354671

hi hyperion

DRIFT_DUR is the number of seconds for the RTC to drift 1 second *faster* than accurate time
to determine DRIFT_DUR i did it this way:
viewtopic.php?f=18&t=4365#p51200

note that if your RTC drift less than 5 minutes faster per month, STM has a register for you to slow down the RTC possibly in an easier and more accurate way
https://www.st.com/content/ccc/resource … 167326.pdf

i wrote this code as my RTC drifts much larger than 5 minutes per month

my RTC drift 10-15minutes per month, so internal RTC calibration not working for me too
Thanks for link. Now i understand how it work

as it seemed i’m not the only one who have lousy 32k crystals, i decided to make this a sketch / demo pretty much with its own ‘mini library’ for those who wants to use this code

you can find the sketch (and mini library) on its own here
https://github.com/ag88/stm32duino_rtcadj
using the ‘mini library’ is somewhat easier and ‘dressed up’ than the raw concepts here, i’ve done the readme.md formatted with markdown on the github web
in a quick summary:

• in setup() call adjtime(); – adjtime() does the adjustments each time the board is reset. But i placed some codes in adjtime() so that the adjustments happen no less than once in 24 hours to reduce cumulative errors
• the function void synctime(time_t time_now); sets the RTC and set the same date / time as the last adjusted date/time. the last adjusted date/time is saved in backup register 8 and 9.
• the function void calibratertc(time_t time_now) calculates the drift duration – the number of seconds for the stm32 rtc to drift 1 secs (faster) and save it in backup register 7

to perform the calibration

• first call synctime(time_t time_now) with the accurate clock time to set the RTC and the last adjustment date / time
• after a day or two call void calibratertc(time_t time_now) with the accurate clock time at the time of calibration and that would compute the drift duration and save it in backup register 7

the (demo) sketch itself has the serial commands to perform the sync and calibration steps, hence you could try that out by simply running the sketch

i’ve tested it on my maple mini but is yet to run it on any other boards.

Additionally, i’ve made a pull request to the STM32F1 libmaple core under STM32F1/libraries/RTClock/examples/RTCAdj
https://github.com/rogerclarkmelbourne/ … 2/pull/581

Thanks

I will take a lo I at the PR

Using the method in the AN2604 application note referenced a few posts ago, it should be straightforward to enable the 512 hertz output and measure the period using the 72 mhz oscillator. 1/period = frequency. 512hz is 1.953 milliseconds. That would give you an idea how bad your crystal really is. A few pf on each side of the crystal to ground might get it back to an acceptable tolerance.

off-topic and some whimsical entertainment

*cheap* 32k crystal may be made from *food* not *quartz*
https://rimstar.org/materials/piezo/how … rystal.htm

now we could guess why they drift so much from the *real* quartz crystals

if this guess is true, the common cheap PZT piezo may produce (far) better (stable) and more accurate oscillations than the 20 for a dollar 32k crystals
https://en.wikipedia.org/wiki/Lead_zirconate_titanate
https://www.aliexpress.com/wholesale?ca … Text=piezo
my guess is the 8mhz crystal that stm32 main clock runs on is likely PZT ceramic based. the only trouble is that it can’t run off VBAT
if your application doesn’t need it to run off VBAT, using HSE for RTC is possibly a good alternative

I think I remember a post in which a very strong drift was caused by the pure presence of the header pins on PC14/15.
So maybe removing them improves already a lot.

oops fixed some bugs and in addition i messed with the hardware BKP_RTCCR adjustments as documented in
AN2604 STM32F101xx and STM32F103xx RTC calibration

now the function void calibrate(time_t time_now) first compute the drift duration and if the drift duration (which is basically the error) is less than 1 in 1048576 / 127 ~ 1 in 8256 secs ~ 314 secs (5 minutes) per month, i simply update BKP_RTCCR and let the hardware do the adjustments
if it is more than that 5 minutes per month, i fall back to my software adjustments

updated in both the pull request
https://github.com/rogerclarkmelbourne/ … 2/pull/581
and the standalone repository
https://github.com/ag88/stm32duino_rtcadj
did a round of tests on maple mini

hopefully with all that maximal adjustments even lousy 32k crystals made from food may pretend to be accurate at least for a while

[ag123 – Mon Dec 10, 2018 6:27 pm] –
…
hopefully with all that maximal adjustments even lousy 32k crystals made from food may pretend to be accurate at least for a while

I really was not prepared for that statement. Ever since this cheap-crystal thread started, I have been distraught. .. I looked around my lab and could not find even one (1) cheap-ass x-tal. That got me thinking; what kind of person would only buy decent crystals? Am I an x-tal elitist? I may need counseling: I feel just terrible. Last night I had nightmares over this condition; I dreamed I won the Powerball lottery for $900M and put it all in a trust to buy decent crystals for the needy. In that dream, I was awarded the Nobel prise for humanitarian efforts. I woke in a cold-sweat.

Ag123 please, please, please buy some decent crystals. PM me if you need and I will mail you a breeding pair so you will never be without your own personal supply. You’ll sleep better. I’ll sleep better.

Ray

hi Ray,

don’t worry too much about this, this whole ‘adjustment’ affair is an overzealous attempt to try out what is possible on stm32
it is interesting that stm32f1x has all the features for one to do all that ‘adjustments’ despite being a rather ‘small’ soc

i’d probably try out one of those epson txco to see what a *real* *quartz* crystal really works like
but i do agree that the *cheap* crystals is probably not even worth its price if it is true that they are after all made from rochelle salt

this detour and that ‘epson crystal factory’ video probably explains why some of the most accurate quartz watches come from japan
https://www.watchreport.com/the-most-ac … the-world/

Ray,

me too think that this RTC drift trick is just an exercise of what can be done with the cheap crystals. I bought them but they are still in the plastic bags because I too think that a good RTC, maybe like the DS3231, is cheap enough so that these crystals are useless.
I am so lazy that I prefer to use NTP on a ESP8266 or any connected boards since I get automatic adjustment when switching to or from daylight saving time, when I have to adjust all old wrist watches, cameras, and other devices that does not switch theirselves. I am switching also to radio controller wall clocks and alarm clocks for the same reason.

RTC DS3231 too big in for some applications. So adjusting “bad” crystal is solution for me I not really fully tested this code yet. But excel say it work

I have been on the look out for one of the older “twin quartz” Seiko watches for a while.
(Watches are a bit of a minor obsession of mine from time to time ).

The Seiko SQ Twin Quartz, with 9723A quartz movement is good for +/- 20s per year, and some of the older, less well known Seiko “twin quartz” models turn up at reasonable prices on fleabay occasionally.

In the mean time (how many times can I shoehorn the word time in to this post I wonder)…. I’ll resist the urge to splurge an eye watering chunk of my month’s salary on the Citizen Chronomaster and content myself with some of the more interesting and affordable items I’ve collected over the years.

After all, where else can you find a kinetic energy powered mechanical analog computer with built in 6Hz oscillator, capable of displaying the result of month long linear computations with an accuracy of around 1 part in 10,000

nice watche(s) who knows one day you may land with a piece you picked up for an ordinary price and auctioned on sotheby’s for millions
it seems the citizen chronomaster might be deemed a *luxury* timepiece
https://www.hodinkee.com/articles/the-c … ab9000-61e
i can’t even find it on the manufacturer’s product home site
https://www.citizenwatch.com/
oh, u’d need to go to the japanese home site to find it
https://citizen.jp/product/the-citizen/ … AB9000-61E
oh and it seemed citizen is going to introduce solar powered editions for it and this century edition watch is *not for sale*
https://www.hodinkee.com/articles/citiz … ntroducing

back on topic i did the ‘adjustments’ on my MM for pretty much the same reasons as hyperion mentioned, the adjustments are adequate to keep things in line for some time and i could simply sync that up to my pc when i hook up via usb again. i think most OS including windows, linux, macos etc are synced to ntp these days and pc time are normally accurate time
it is good that stm32 has good RTC functionalities built-in, i think not many/all mcus has it

just an update, it seemed that out of an accident, i got some parts per million accuracy, a ‘manual’ measurement shows that the maple mini running on a coin cell and a cheap 20 for a dollar 32k crystal and no caps (yup 0pf, it is oscillating like an uncontrolled flywheel/pendulum) is in sync with my pc synced to ntp after 15 days – 0 secs drift – that is after all that ‘adjustments’ – done by the code
but i guess this is *stochastic*, the drifts may come back again, it so happen that it is hitting a mean of zero for now

the drift duration (i.e. error for this crystal is extreme without the caps) 1/2200, if my maths is ok, i think that makes it 32668 + 1/2200 ~ 32768 + 15 = 32782 hz. in each second, it drifts 15 parts of 32768 faster

perhaps that means a crystal made of food (rochelle salt) works as well, perhaps it won’t be as stable or last as long as a real quartz crystal
incidentally without those caps, it is a blessing in disguise, it makes the crystal easier to oscillate, but i’d guess it’d be less stable and possibly drift when the ‘environment’ changes. i think the drift is possibly a manufacturing precision issue, i.e. every crystal is cut slightly different and is not tuned and a ‘large’ cap (e.g. 10pf) is specified to ‘pull’ the frequency lower, making do without that capacitor may after all be a good thing

thanks ray
it seemed quite possible that without the caps the crystal locks on to the parallel resonance or even anti resonance frequencies high impedance near zero currents hence the fixed drift to a higher frequency, my very rough/naive understanding of some of the physics
https://electronics.stackexchange.com/q … nce-graphs

Also fooled around with using the PPS of a GNSS to crosscorrect the alignment of the RTC.

I played with something like that: a GPS-disciplined VC(TC)XO, quite successfully. the basic idea is to use the GPS to gate a VCXO’s output. From that you can determine timing error, which drives a PWM generator, via a control algorithm, that is linked to the VCXO’s control voltage via a LPF – or a DAC if your chip happens to have it.

The beauty of it is that you can 1) calibrate a TCXO/VCXO, and 2) you can generate lots of weird frequencies unsupported by a GPS.

The MCU essentially performs the function of a frequency meter (using the GPS signal as a reference clock) and a voltage generator. It should be easily ported to the Arduino / STM32 environment – I originally wrote it for AVR/PIC.