sprintf function uses a lot of memory!

rexnanet
Sun May 15, 2016 1:20 pm
Hi,
I’m trying to use OBD and ILI9341 libraries at the same time but ran out of code memory.
Using only ILI9341 I’m using about 40kB but it’s normal because of the fonts and stuff.
But OBD library alone is also using more than 40kB and I found it a bit overkill considering it’s only using serial port and some string manipulation.

So I went in a search to find who was the guilty.
While stripping the OBD library code I found the culprit! sprintf!!!! :shock:

sketch_sprintf.jpg
sketch_sprintf.jpg (83.05 KiB) Viewed 1532 times

martinayotte
Sun May 15, 2016 3:20 pm
That is because the libc implementation brings the whole math implementation for floats.
You can do a local libc-replacement, a bit like it has been done for other MCUs, even avr-libc, in such way that if floats printf are not needed, it will be quite small.

stevech
Sun May 15, 2016 5:33 pm
and 64 bit integers

rexnanet
Wed May 18, 2016 4:00 pm
martinayotte wrote:That is because the libc implementation brings the whole math implementation for floats.
You can do a local libc-replacement, a bit like it has been done for other MCUs, even avr-libc, in such way that if floats printf are not needed, it will be quite small.

sheepdoll
Wed May 18, 2016 4:49 pm
This came up in the HALMX thread too as many implementations of itoa call sprintf.

martinayotte
Wed May 18, 2016 7:09 pm
rexnanet wrote:
I’ll have to recompile the whole toolchain for that, right? I think maybe it will be a bit overkill…

Slammer
Wed May 18, 2016 9:33 pm
Take sprintf from here: http://www.stm32duino.com/viewtopic.php?f=18&t=1014
A couple months ago I made a research about small printf functions and this implementation is the most lean with full support of integers (8/16/32bit), width specifiers and strings (no fp). The discussion was about Serial.printf but inside there is a custom sprintf (in the first version). It is very small, a bit larger than 1KB.

rexnanet
Fri May 20, 2016 9:06 am
Thanks! This is really better than having to compile the whole thing.

I found out that other functions also make the same “enlargement” effect, atof().
I’ll try to find an alternative for this one also.


Slammer
Fri May 20, 2016 1:43 pm
During my research about small printf functions, I found some tricks to remove almost completely the standard c functions for floating point arithmetic, while keeping the %f support for sprintf.
It is better to adopt the sprintf function from SDCC compiler (compiler for small 8bit MCUs like 8051, Microchip, STM8 etc). Look the source of their “standard C library” for printf. As I remember, it is possible to support floating point with 1.5-2 KB more code memory.

rexnanet
Sat May 21, 2016 6:18 pm
I’m having some issues while implementing this.

I’ve managed to get one version of sprintf compiling in a sketch. I’m still to test it to see if it works properly but seems ok in terms of space (12kB is only 1kB more :) ).

But how can I get the libraries I use to use this version? I can get the loop() function to use it but externally how can it be done?
I’m not very familiar still with the Arduino code architecture. Can I add this new code into the Print class and make a “sprintf2” function that the libraries can use instead?
I’ve tried to implement the “lean Serial.print()” but I think the Arduino “IDE” its not compiling the Print.cpp… And if it includes an external .c file, it should also compile it, but noting…
I saw the rules.mk file and it should do that but something is missing…at least to my eyes :)


rexnanet
Sat May 21, 2016 6:40 pm
A small step… :)

I’ve created sprintf2.c and .h and placed them on STM32F1/cores/maple and it compiles, but I get the warning:

C:\Users\Rui\AppData\Local\Temp\build5420264372001361322.tmp\zzztest.cpp.o: In function `loop()’:
G:\User\Downloads\arduino-1.6.5-r5/zzztest.ino:196: warning: undefined reference to `sprintf2(char*, char const*, …)’

I’ve added the #include “sprintf2.h” in the sketch so it shouldn’t be doing this…
Code size got reduced so I’m gessing the .o file wasn’t added to the binary, although it compiled and .o was generated…

The sketch:

#include "sprintf2.h"

char buff[100];

void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
}

void loop() {
// put your main code here, to run repeatedly:
sprintf2(buff, "%d", millis());
//char *p = strstr(buff, "0: 49 02");
//atof(buff);
Serial.print(buff);
delay(1000);
}


martinayotte
Sat May 21, 2016 9:18 pm
If the error is at link time, probably due to C/C++ signature differences :

extern "C" {
#include "sprintf2.h"
}


rexnanet
Sun May 22, 2016 4:51 pm
martinayotte wrote:If the error is at link time, probably due to C/C++ signature differences :

extern "C" {
#include "sprintf2.h"
}


GrumpyOldPizza
Tue May 24, 2016 6:58 pm
rexnanet wrote:Hi,
I’m trying to use OBD and ILI9341 libraries at the same time but ran out of code memory.
Using only ILI9341 I’m using about 40kB but it’s normal because of the fonts and stuff.
But OBD library alone is also using more than 40kB and I found it a bit overkill considering it’s only using serial port and some string manipulation.

So I went in a search to find who was the guilty.
While stripping the OBD library code I found the culprit! sprintf!!!! :shock:

Removing the single line where sprintf is called and the size of the code drops by around 20kB!!! :shock:
sketch_blank.jpg


stevech
Wed May 25, 2016 3:24 am
I would think that GCC in Arduino has, like IAR and Keil, choices in the IDE or command line for which flavor of printf (actually, vprintf which is IO stream independent), is used. Mine has about 4 choices, such as
omit floating point
omit 64 bit integers
omit some obscure format specifier options
use minimal, e.g., char, int, signed and unsigned.
These are precompiled .a files libraries; so too, Arduino, if they do offer choices as above, as they surely do, else small MCUs would choke.

Also same concept, but for scanf() and vscanf()


rexnanet
Wed May 25, 2016 1:24 pm
GrumpyOldPizza wrote:
Arduino Zero uses –specs=nano.specs for recipe.c.combine.pattern in platforms.txt. This will use newlib-nano, which in turn has a default optimized printf/scanf implementation that gets rid of floats and other less needed cruft. As a side-effect it also reduces the RAM foot print substantially by using less memory for impure data.

GrumpyOldPizza
Wed May 25, 2016 1:34 pm
rexnanet wrote:GrumpyOldPizza wrote:
Arduino Zero uses –specs=nano.specs for recipe.c.combine.pattern in platforms.txt. This will use newlib-nano, which in turn has a default optimized printf/scanf implementation that gets rid of floats and other less needed cruft. As a side-effect it also reduces the RAM foot print substantially by using less memory for impure data.

Slammer
Wed May 25, 2016 2:42 pm
The linker specifier –specs=nano.specs working also with STM32
I am building the programs with makefiles and the code is about 2K-3K smaller (GrumpyOldPizza is correct) and the printf family routines are also smaller (about 10KB smaller) but not like a custom printf/sprintf. Some programs have problems with nano.specs but generally is OK.
I can’t see a reason for using printf from library (either normal or nano), you can use a custom version with much smaller footprint dedicated to your application.

GrumpyOldPizza
Wed May 25, 2016 3:09 pm
Slammer wrote:I can’t see a reason for using printf from library (either normal or nano), you can use a custom version with much smaller footprint dedicated to your application.

Slammer
Wed May 25, 2016 3:38 pm
It depends on how badly you need the code memory.
For example, when playing with F401RE, with 512KB flash, there is no need to strip down anything (in 99.9% of cases), but in a project with small TSSOP mcus (eg. STM32F031F4P7) every byte counts.
Anyway, the code memory is cheap these days, specially with ARM mcus.

stevech
Wed May 25, 2016 5:10 pm
Slammer wrote:It depends on how badly you need the code memory.
For example, when playing with F401RE, with 512KB flash, there is no need to strip down anything (in 99.9% of cases), but in a project with small TSSOP mcus (eg. STM32F031F4P7) every byte counts.
Anyway, the code memory is cheap these days, specially with ARM mcus.

GrumpyOldPizza
Wed May 25, 2016 6:33 pm
stevech wrote:Slammer wrote:It depends on how badly you need the code memory.
For example, when playing with F401RE, with 512KB flash, there is no need to strip down anything (in 99.9% of cases), but in a project with small TSSOP mcus (eg. STM32F031F4P7) every byte counts.
Anyway, the code memory is cheap these days, specially with ARM mcus.

stevech
Wed May 25, 2016 9:12 pm
I’m in the world of $20 STM32F4’s. And with a good IDE and debugger, which per me, isn’t Arduino’s IDE by a long shot.

Just like any hobby is more fun with good tools and equipment. Better as time goes by.

Trying to get crap-China down to $5-10 is sheer masochism when you try to USE the junk, to include no documentations.

If I were a university student now, not knowing any better, I’d appreciate being told that for less than the price of a pizza, the risk and difficulty of using a $20 board instead of a $5 will likely help me me get an A and turn project in on time. And make me a hero to the project team. Same principle applies to hobby work.

Sadly, all too often, the professors have nada experience in industry and the real world outside the bubble of academia where time is valuable.


Slammer
Wed May 25, 2016 9:43 pm
As I am a newbie in STM32 world, my first STM32 board was the Nucleo-401RE ( I had an olimexino maple clone for many years but never touched) bought from a local store for 14 euro. But the curiosity about what a 3$ board can do, make me to play with the BluePill.
I agree with you, a “big” MCU with correct documentation is the best way for someone to start, the big nucleo boards offer exceptional value.

mrburnette
Wed May 25, 2016 10:36 pm
stevech wrote:I’m in the world of $20 STM32F4’s. And with a good IDE and debugger, which per me, isn’t Arduino’s IDE by a long shot.

Just like any hobby is more fun with good tools and equipment. Better as time goes by.

Trying to get crap-China down to $5-10 is sheer masochism when you try to USE the junk, to include no documentations.
<…>


stevech
Wed May 25, 2016 10:59 pm
To each his own.

RogerClark
Thu May 26, 2016 2:20 am
@slammer

Spend $5 and buy a Maple mini.

These boards are much higher quality than the the $2 or $3 Blue Pill boards and have less problems with USB

@stevech

The whole point of this forum is for people who want to use Arduino on STM32. There are other forums for people who want to use other tools.

Although I’m in favor of free speech, your appear to be trying to drive people off the forum and away from the Arduino ethos


rexnanet
Fri May 27, 2016 1:04 pm
STM32F4’s can also be used in Arduino mode, right? :) (Although not fully supported if I got it right from what I read)

@stevech has got some point on Arduino IDE, it a big piece of… I only use it to compile and send the code to the STM32. Other than that even notepad++ is far more usefull than it! I don’t know how they call it “IDE”… :?

But even with F1’s we can still make stuff work :) and at a fraction of the price, which is very good when making lots of different projects.


RogerClark
Fri May 27, 2016 2:29 pm
I also just use the IDE as a compile / upload tool.

I use Notepad++ on Windows as my editor, as it is a much better editor than the IDE.

I view Arduino as the whole ecosystem, e.g. IDE as compile and upload tool, Core API and Core libs (SPI I2C etc), and the masses of third party libs and other code e.g. display driver libs, as well as whole suites of code like Arduino JSON, and Open Energy Monitor etc etc etc

Even if I’m not developing for Arduino, I often find that there is an open source Arduino lib that has code that I can use.

I realise that not all Arduino code libs are good quality, but my code is a buggy as the next person’s, and getting pier reviewed and corrected code libs from github, is on balance, normally going to be more stable less buggy, than me trying to write code from scratch based on device data sheets

But, I always look at how active a github repo is, and how recently its been updated and how many forks its has, to get a feel for what the stability etc is likely to be.
As on the flip side, in my day job, we have bought in various pre-written closed source packages of code (albeit small and cheap packages), and the code quality is highly variable. (this is often as the code has been written for a specific project then resold as general purpose, and its hard to re-purpose to make small changes).
Sometimes the code is just poor quality, but thats the luck of the draw when buying in closed source code, with no way to see it all before you pay for it.


mrburnette
Fri May 27, 2016 11:42 pm
RogerClark wrote:
<…>
As on the flip side, in my day job, we have bought in various pre-written closed source packages of code (albeit small and cheap packages), and the code quality is highly variable. (this is often as the code has been written for a specific project then resold as general purpose, and its hard to re-purpose to make small changes).
Sometimes the code is just poor quality, but thats the luck of the draw when buying in closed source code, with no way to see it all before you pay for it.

rexnanet
Sun May 29, 2016 9:32 am
GrumpyOldPizza wrote:
Arduino Zero uses –specs=nano.specs for recipe.c.combine.pattern in platforms.txt. This will use newlib-nano, which in turn has a default optimized printf/scanf implementation that gets rid of floats and other less needed cruft. As a side-effect it also reduces the RAM foot print substantially by using less memory for impure data.

stevech
Sun May 29, 2016 8:33 pm
RogerClark wrote:@stevech

The whole point of this forum is for people who want to use Arduino on STM32. There are other forums for people who want to use other tools.

Although I’m in favor of free speech, your appear to be trying to drive people off the forum and away from the Arduino ethos


Leave a Reply

Your email address will not be published. Required fields are marked *