I strongly disagree that dynamic allocations should be avoid in EVERY cases, it should not become religious bevhaviour !
Many and many Arduino libraries and even core classes such String are using malloc/realloc/free, even in AVR328.
So, please, don’t be scared of using those if managed properly !
I suggest to everyone grepping their own libraries folder to figure out that is true !
I strongly disagree that dynamic allocations should be avoid in EVERY cases, it should not become religious bevhaviour !
Of course not in every case!
But, it is like giving the car keys to a teenager… kind of stuff that will keep a parent up at night. Use should be tightly controlled, stringently tested, and vetted by by as many eyes as possible!
Ray
I agree with you on the EndUser side where newbies are building link-list and forget to implement free-node …
But when talking about core files such as Serial buffers, where those buffers would be allocated in indirectly Setup() via Serial.begin() and that those buffers are alive forever (in those case, Roger reluctance about fragmentation won’t occur), there is not much issue there compared to String alloc/free shuffling or Ethernet library shuffling .
That is the goal of this thread : “Removing the myths about Dynamic Allocations” in Embedded.
As said in the original post, why even STM in their SPL is using malloc/realloc/free in their code if it is a “bad practice” ?
In other words, “Dynamic Allocations problems in Embedded world” are Myth or Reality ? (for me it is a “myth”)
<…>
In other words, “Dynamic Allocations problems in Embedded world” are Myth or Reality ? (for me it is a “myth”)
I searched libmaple and could not find anywhere that Leaflabs used malloc
We don’t use new at all because it pulls in a huge amount of extra libraries which causes the code size to balloon massively.
I’ve taken a look at STM’s own HAL and CMSIS and I can’t find a single reference to malloc
I did the same on libopencm3 https://github.com/libopencm3/libopencm … 3&q=malloc
and couldn’t find any code references to malloc (only one a perl file), the same applies to the new operator.
In the case of the Serial buffer, there is no need to use malloc, the buffers can be statically allocated passed into Serial.begin() – albeit the buffer size(s) need also to be passed as otherwise the code has no way to know how big the buffers are.
<…>
I searched libmaple and could not find anywhere that Leaflabs used malloc
I love a good discussion! I suits my “wishywashy” personally so well
In school, I would always take the Devil’s Advocate position… often switching the Devil’s role midstream just like a politician
It is great learning but did not make for great popularity to my peers… folks do not know what to make of someone that they cannot pin a label upon.There are custom malloc functions for replace the bloated full fledged versions of standard libraries.
Yesterday I submitted PR #184 to apply some USART improvements in the F1 library.
One of these improvements is dynamic buffer allocation which I implemented using malloc().
This is concerning Roger who is inclined for a different approach in which the user should be able to allocate the buffer and pass it (together with its size) to the Serial device.
On the other hand I have concerns with his approach that I listed in this post: viewtopic.php?f=3&t=1189&start=50#p15662
So, I’d like your opinion on following points:
- do you think my usage of malloc() is (not) a case of fair usage?
- what do you think about my concerns on the approach suggested by Roger?
Thanks in advance and best regards, E.
Personally, I would rather just replicate what the AVR does and declare static non-optional buffers, but I know that would chew up 4 x 64 bytes of RAM that some people dont want to loose
@martin
Thanks. I didnt look for alloc. My mistake.
@slammer
Good point.
What do we do in Serial.end(), call free() on those buffers?
I think I need to put this to the pole as my personal opinion isn’t that important.
I only want whats best for the core, but ultimately its the community who are the arbiter of whats best.
Just so long as its not another Brexit vote style vote 
Just so long as its not another Brexit vote style vote
<…>
I think I need to put this to the pole as my personal opinion isn’t that important.
I only want whats best for the core, but ultimately its the community who are the arbiter of whats best.
<…>
In 99.99% of the cases, Serial object are allocated globally (but even if they aren’t the following still true), it’s constructor is initializing the buffers pointer to NULL, those pointers will get set during Serial.begin() according to buffer size requested, since original pointers are NULL, realloc() will call malloc() (like in the String object mentioned earlier). Those buffers should remained allocated until the Serial destructor been called (not related with Serial.end() at all).
In all the above scenario, there is no heap fragmentation at all.
If a new Serial.begin() is been called on existing Serial object, then the buffers pointer initializing should check if they are NULL but also if buffer sizes differ from the the original sizes, in which case realloc() will be done accordingly.
This is the only case where heap fragmentation can occur !
Is it really worth to think about multiple Serial.begin() behaviour, even if properly done, especially that this is even unknown on AVR ?
(We have another thread about multiple Serial.begin() behaviour, but here if buffer sizes are identical, it won’t be an issue)
I think there is another thread where its been noted that calling Serial.begin() again, and not calling Serial.end() first may not work anyway.
But if malloc is called from Serial.begin() we should call free in Serial.end() just to be consistent.
I suppose in Serial.begin() if the pointers are non-null, free could be called on the pointers, or possibly realloc()
So. I get the feeling that as we’ve had very few people comment that no one is too fussed about this and that it should be implemented as is.
Well, except the buffer size params should be at the end of the argument list
But one other thing puzzles me, I thought libmaple already had a read buffer
wouldnt this change make the read buffer optional as well?
Or am I missing something ?
I think there is another thread where its been noted that calling Serial.begin() again, and not calling Serial.end() first may not work anyway.
But if malloc is called from Serial.begin() we should call free in Serial.end() just to be consistent.
I suppose in Serial.begin() if the pointers are non-null, free could be called on the pointers, or possibly realloc()
This way I think nothing will be improved..
Features/advantages of the solution I proposed are:
- buffered interrupt-based USART TX (this can significantly speed up TX response time)
- runtime buffer allocation on the only USART Serials effectively “beginned” (the buffer is anyway effectively allocated using malloc() in the ring_buffer “rb_init()” and not in the Serial.begin())
- no need to allocate default buffer on non-used USART Serials
- deallocation of previously allocated buffer (if any) in case rb_init is called more than once on the same ring buffer
- support for non-blocking TX (blocking is default)
Possible drawbacks:
- heap fragmentation in case the same Serial is beginned several times with different buffer sizes
- rb_init() interface change: this requires code reworking in users code that explicitly use ring_buffer -> I’m just thinking on this and I think I should make it backward compatible
- no reuse of the same buffer in case rb_init is invoked on an existing ring buffer with the same buffer size -> easy fix
- don’t know how to handle possible malloc() failures, any suggestion?
@Roger: buffers (for beginned devices) are not optional, what’s optional is only setting their dimension which must be >=1 for TX buffers and >=16 for RX buffers (if less specified I default to minimum).
-ffunction-sections -fdata-sections at compile and -Wl,–gc-sections at link.
Looking at the compiled code they sure seem to be there.
-rick
-ffunction-sections -fdata-sections at compile and -Wl,–gc-sections at link.
Looking at the compiled code they sure seem to be there.
@devan
But here in this thread, we would also want you give your opinions about basic dynamic allocations such the one currently found in WString.cpp and if this could be applied to Serial buffer as is.
So, please give you opinion !
(personally,I don’t have any objections, but this thread is to make sure that community democracy is respected …)
Given that the Arduino/Wiring/Wirish way is to implicitly include things so that the average user doesn’t have to ask for them, I agree with Roger that we should just allocate the buffers by default. I think that the users that are concerned with using all 20KiB of SRAM are knowledgeable enough to manually apply the patches needed to claw back the 2-3 KiB used by a blank sketch.
edogaldo, I think there’s plenty of value in a Serial buffer that interoperates with dynamic memory allocation. However, my general gut feeling is that less unoverrideable dynamic allocation is better (I’ve overridden malloc and free with no-ops save space on a previous non-Arduino project). Of course, using static buffers introduces unoverrideable static allocations, so it’s really a matter of preference.
I suspect that the majority of users will not run into any problems with either implementation.
One comment about malloc though, if you are going to malloc things, it is better to over allocate and use the same size malloc requests so that it can reclaim freed memory more easily. So instead of allocating 57 bytes and 62 bytes use a 64 byte malloc request for both of them. This will reduce heap fragmentation.
-rick
oh yeah and like devan I’m really not a stakeholder here.
