Blue Pill Serial.print slow, UNO faster?

danSTM
Tue Jun 21, 2016 12:43 pm
Greetings All,

I can’t seem to figure out why this code takes 6617us on the blue pill and takes 2472us on an UNO. I’ve tried both Serial and Serial1 and get the same results.

I’m using serial upload with the built-in loader. IDE is 1.6.9 and core is up to date.

Code generally runs 7x faster, until I hit “Serial.prints”.

Thanks for any feedback!

//#define Serial Serial1 // TX A2

byte nodeID = 10;
byte packetNum = 11;
float C = 42.611;
float F = 42.611;
float H = 42.6;
float vBatt = 3.14;
int packetLoss = 10;
int packetTotal = 10000;

void printData() {
Serial.print(nodeID);
Serial.print(",");
Serial.print(packetNum);
Serial.print(",");
Serial.print(C);
Serial.print(",");
Serial.print(F);
Serial.print(",");
Serial.print(H);
Serial.print(",");
Serial.print(vBatt,3);
Serial.print(",");
Serial.print(packetLoss);
Serial.print(",");
Serial.println(packetTotal);
}

void setup() {
Serial.begin(57600);
delay(3000);
unsigned long start = micros();
printData();
unsigned long end = micros();
Serial.println(end - start);

}

void loop() {

}


Slammer
Tue Jun 21, 2016 2:07 pm
This is a result of polling nature of transmit function in arduino_stm32.
As the core does not support interrupt on transmit, MCU must wait for every byte before sending another (polling transmit).
UNO has interrupt based transmit with ring buffer, the Serial.print function simply pushes the bytes to ring buffer and the interrupt handler sends the bytes.
PS:I hope that this will change in the near future…

simonf
Tue Jun 21, 2016 2:27 pm
I modded your code so I could see what effect baud rate has on the speed.

#define SerialOUT Serial1 // TX A2

byte nodeID = 10;
byte packetNum = 11;
float C = 42.611;
float F = 42.611;
float H = 42.6;
float vBatt = 3.14;
int packetLoss = 10;
int packetTotal = 10000;

void printData() {
SerialOUT.print(nodeID);
SerialOUT.print(",");
SerialOUT.print(packetNum);
SerialOUT.print(",");
SerialOUT.print(C);
SerialOUT.print(",");
SerialOUT.print(F);
SerialOUT.print(",");
SerialOUT.print(H);
SerialOUT.print(",");
SerialOUT.print(vBatt,3);
SerialOUT.print(",");
SerialOUT.print(packetLoss);
SerialOUT.print(",");
SerialOUT.println(packetTotal);
}

void setup() {
Serial.begin(57600);

//SerialOUT.begin(300);

}

void loop() {

unsigned int Baudrates[]= {300,1200,2400,4800,9600,19200,38400,57600,115200};

unsigned long start;
unsigned long end;

for ( int ind =0 ; ind < 9; ind++ ){

Serial.print("AT Baudrate it Tates: -");
Serial.print(Baudrates[ind]);
Serial.print(" , ");
SerialOUT.begin(Baudrates[ind]);
start = micros();
printData();
unsigned long end = micros();
Serial.print(end - start);
Serial.print("uS. To send 1 char it should take uS :");
Serial.println((10000000/Baudrates[ind]));

delay(3000);
SerialOUT.end();

}

}


danSTM
Tue Jun 21, 2016 2:54 pm
Thanks for the feedback!

The lack of the interrupt and buffer is certainly a problem for me and probably for others.

I use RF receivers that use a timer interrupt and I don’t begin polling that interrupt again until I process the incoming data and output it as csv.

I thought I’d give the STM a try. The faster I leave the process data function, the sooner I can get back to polling the ISR. The gain in processing the data is negated by the blocking prints.

+1 for the buffer.

Roger mentions it here http://www.rogerclark.net/updates-to-ar … tm32-code/


simonf
Tue Jun 21, 2016 8:13 pm
Slammer wrote:This is a result of polling nature of transmit function in arduino_stm32.
As the core does not support interrupt on transmit, MCU must wait for every byte before sending another (polling transmit).
UNO has interrupt based transmit with ring buffer, the Serial.print function simply pushes the bytes to ring buffer and the interrupt handler sends the bytes.
PS:I hope that this will change in the near future…

danSTM
Thu Jun 23, 2016 11:59 pm
…how can we get this effort going? …it appears to me that this isn’t that easy.

..I’ve looked at https://github.com/arduino/Arduino/tree … es/arduino

..and https://github.com/esp8266/Arduino/tree … es/esp8266

Thanks!


RogerClark
Fri Jun 24, 2016 1:14 am
I think that @ekawahyu is looking at this for the HAL MX core but is having some problems with it (see)

viewtopic.php?f=3&t=1012&start=60#p15108

I think the other small problem is that some people would not want to loose any RAM that would need to be allocated to all the hardware serial devices.

i.e The STM32F103C has 3 hardware serial devices. All of which would allocate the buffer.

I know its only 192 bytes, but I’m not sure if everyone wants to trade that for Serial performance.
Though I suppose we can have a Poll on the forum to find out.


mrburnette
Fri Jun 24, 2016 12:54 pm
RogerClark wrote:<…>
I know its only 192 bytes, but I’m not sure if everyone wants to trade that for Serial performance.
Though I suppose we can have a Poll on the forum to find out.

simonf
Sun Jun 26, 2016 6:04 pm
I had a look into this myself a blocking serial transmit would mess up some of my projects and explains the reason why I had to use an interrupt driven routine on one of my other projects to stop it being laggy when it was communicating over serial.

Its not a good idea to allocate a fixed 3*64byte buffer as has been said, especially when you may only be using 1 port.

I did some experiments and found that if you send one character the serial port does not block unless its already sending. So it should be possible to create a Virtual Serial Port with a ring buffer with a timer or even better an “interrupt on transmission finished” that feeds the real com Port. A #define for the buffer size would be good then you just define as many buffers as you need of the size you need (Probably a couple of bytes bigger than the largest message you intend to send).


Slammer
Sun Jun 26, 2016 8:07 pm
The correct way is the “interrupt on transmission finished”, as you said….
If we keep each HardwareSerialX class separately it is possible to allocate the buffers only for Serial ports that are used in the project. I think that 64 bytes for transmit buffer is rather big, especially for fast speeds, but it depends on the application. We can set the default buffer size to 16 bytes (like arduino UNO) and if someone wants to change this, a simple DEFINE before include “HardwareSerialX.h” can do the job…. without breaking arduino API.

simonf
Sun Jun 26, 2016 10:43 pm
I have written the ring buffer and tested it. My development environment, VS2012 is broke since the upgrade to 1.6.9 so I am having to work in the arduino environment which is a little painful for me as c++ is not my favoured language. By the time I get used to it, my project is finished and I forget it for another 6 months.

edogaldo
Mon Jun 27, 2016 2:55 pm
My tip: can’t say about the buffer but about using the interrupt I think the FreeRTOS example can support:

void USART1_IRQHandler( void )
{
long xHigherPriorityTaskWoken = pdFALSE;
char cChar;

if( USART_GetITStatus( USART1, USART_IT_TXE ) == SET )
{
/* The interrupt was caused by the THR becoming empty. Are there any
more characters to transmit? */
if( xQueueReceiveFromISR( xCharsForTx[ 0 ], &cChar, &xHigherPriorityTaskWoken ) ) // take the char to be sent from the FreeRTOS queue
{
/* A character was retrieved from the buffer so can be sent to the
THR now. */
USART_SendData( USART1, cChar );
}
else
{
USART_ITConfig( USART1, USART_IT_TXE, DISABLE );
}
}

if( USART_GetITStatus( USART1, USART_IT_RXNE ) == SET )
{
cChar = USART_ReceiveData( USART1 );
xQueueSendFromISR( xRxedChars[ 0 ], &cChar, &xHigherPriorityTaskWoken );
}

portEND_SWITCHING_ISR( xHigherPriorityTaskWoken );
}


edogaldo
Tue Jun 28, 2016 7:43 pm
Buffered_InterruptBased_USART_TX.zip
(22.12 KiB) Downloaded 32 times

Slammer
Tue Jun 28, 2016 8:34 pm
Thank you! I will test it…

edogaldo
Tue Jun 28, 2016 9:23 pm
edogaldo wrote:Buffered_InterruptBased_USART_TX.zip

edogaldo
Wed Jun 29, 2016 9:44 pm
edogaldo wrote:edogaldo wrote:Dear all, I tried to apply buffered interrupt based USART TX on STM32F103 devices, the ones I know (more or less :mrgreen: ).
Here the modified files:
– Arduino_STM32-master\STM32F1\system\libmaple\include\libmaple\usart.h: added ring_buffer wb in struct “usart_dev”
– Arduino_STM32-master\STM32F1\system\libmaple\usart_private.h: modified interrupt handling function to handle tx
– Arduino_STM32-master\STM32F1\cores\maple\libmaple\usart.c: modified usart_tx() to use wb if TXE not ready
– Arduino_STM32-master\STM32F1\cores\maple\libmaple\usart_f1.c: modified USARTs definition to add new tx buffer “wb”

I attached also the corresponding original versions I had as I don’t know if they have been modified since when I got them (more or less 1 month ago).

Hope this helps..

PS: could not test it on a real device but at least it compiles successfully..

Best, E.


RogerClark
Wed Jun 29, 2016 9:47 pm
excellent!

I think this is definitely something that should be put into the core.

I will test it at the weekend and do a separate post / poll to see what the community things.

Thanks again

Roger


edogaldo
Wed Jun 29, 2016 9:52 pm
I like this community, it’s been my pleasure.

RogerClark
Wed Jun 29, 2016 9:58 pm
Thankyou so much.

I am sure everyone will be very appreciative, as we all want the core to work as well as possible


edogaldo
Wed Jun 29, 2016 10:41 pm
Slammer wrote:The correct way is the “interrupt on transmission finished”, as you said….
If we keep each HardwareSerialX class separately it is possible to allocate the buffers only for Serial ports that are used in the project. I think that 64 bytes for transmit buffer is rather big, especially for fast speeds, but it depends on the application. We can set the default buffer size to 16 bytes (like arduino UNO) and if someone wants to change this, a simple DEFINE before include “HardwareSerialX.h” can do the job…. without breaking arduino API.

simonf
Wed Jun 29, 2016 11:01 pm
edogaldo wrote:I like this community, it’s been my pleasure.

RogerClark
Thu Jun 30, 2016 12:04 am
I have a feeling that defines in the sketch do not get picked up by the core code.

I recall trying to do something similar to prevent the SPI class getting instantiated at all, but it didnt work, because those definitions would need to be in a header that is read in my the core.

The only way to control it would either be to add something into the compiler directives e.g. -DUSE_SERIAL_TX_BUFFER=1 and then have an option menu in boards.txt that set flags that were picked up by platform.txt

Or alternatively, overload Serial.begin to add an additional argument to enable TX buffering.

Or the user could just edit one of the core headers e.g. Arduino.h and add the define in there if they wanted to have a TX buffer, in which case we should probably allow the size to be set, e.g.

#define SERIAL_TX_BUF_SIZE 128

and use

#ifdef SERIAL_TX_BUF_SIZE

around the new code

However I suspect the majority of people will prefer to have this even if it means they loose some RAM

I’ve not looked at you code, but what happens if they print more than a buffers worth? I presume it blocks until just 1 buffer’s worth is left, and then continues ?? (Sorry not had chance to review the code yet)

Thanks

Roger


Slammer
Thu Jun 30, 2016 1:53 am
Official Arduino API has fixed values for RX/TX buffers and there is no way for the user to change them in sketch level.
We are talking actually for an improvement/enhancement of Arduino API…
I think that 64 bytes is a very good value, for TX/RX buffers, nobody will miss the lost bytes…. we are in ARM world… (even AVR arduino allocates 64 bytes for TX/RX for Atmega168/328 devices). The most important is to allocate the buffers only for the Serial ports that are used in project.

RogerClark
Thu Jun 30, 2016 4:24 am
I don’t feel that we should be limited by the AVR API, but equally we should not do anything which breaks the API either

Although we could pass the buffer as an additional param to Serial.begin

I think its easier to just have the buffer size defined somewhere that people can get to it if they really need bigger buffers e.g. in one of the top level core headers.

And we should probably set it to the same value that the AVR uses. (as I presume some thought went into that value)

NB. I’ve not looked at the buffer value on the Due or Zero etc, to see if the buffer size is bigger, or whether its implemented at all.


edogaldo
Thu Jun 30, 2016 5:47 am
If I correctly interpreted the code, SAM implementation uses 128 bytes for each RX and TX buffer and allocates them for all available usarts/uarts, ref:
– variant.cpp
– RingBuffer.h

Some more notes about the STM32F1 implementation:
– the buffer size is not only used in rb_init() but particularly in the “usart_dev” struct definition so I’m not surewe can allocate it in a dynamic way keeping the same approach:
/** USART device type */
typedef struct usart_dev {
usart_reg_map *regs; /**< Register map */
ring_buffer *rb; /**< RX ring buffer */
ring_buffer *wb; /**< TX ring buffer */
uint32 max_baud; /**< @brief Deprecated.
* Maximum baud rate. */
uint8 rx_buf[USART_RX_BUF_SIZE]; /**< @brief Deprecated.
* Actual RX buffer used by rb.
* This field will be removed in
* a future release. */
uint8 tx_buf[USART_RX_BUF_SIZE];
rcc_clk_id clk_id; /**< RCC clock information */
nvic_irq_num irq_num; /**< USART NVIC interrupt */
} usart_dev;


edogaldo
Thu Jun 30, 2016 7:48 am
I’ve not looked at you code, but what happens if they print more than a buffers worth? I presume it blocks until just 1 buffer’s worth is left, and then continues ?? (Sorry not had chance to review the code yet)
I based my implementation on the standard Arduino implementation which seems to me quite well optimized, the “write()” code works like this:

if [ready to send] then send
else
while [buffer full] if not ["ready to send" interrupt enabled] then
if [ready to send] then send first buffered char
else do nothing (wait for interrupt to free the buffer)
put input char in buffer
enable "ready to send" interrupt


RogerClark
Thu Jun 30, 2016 9:14 am
edogaldo wrote:…..
so the buffer full is a supported scenario.

Best, E.


mrburnette
Thu Jun 30, 2016 12:01 pm
edogaldo wrote:
<…>
– asking users to modify library files (i.e. asking them to change the value of USART_RX_BUF_SIZE/USART_TX_BUF_SIZE) I think is against the Arduino “MCUs for dummies” spirit
<…>

martinayotte
Thu Jun 30, 2016 12:32 pm
RogerClark wrote:Although we could pass the buffer as an additional param to Serial.begin

mrburnette
Thu Jun 30, 2016 12:40 pm
martinayotte wrote:RogerClark wrote:Although we could pass the buffer as an additional param to Serial.begin

simonf
Thu Jun 30, 2016 1:23 pm
I wrote my own ring buffer code effectively making a virtual com port which in turn fed the real serial port 1 character at a time thus stopping the blocking. I had not figured out the IRQ for feeding it so did so as a timer a very crude fix but a fix none the less.

I must admit I had thought the TX serial was buffered untill I saw this thread but I spotted and understood the problem and am capable of doing a crood fix. I have not done many projects with STM32 but I now undertand why at least one was not being as responsive as I thought it should be compared to the Arduino Pro Micro.

For me TX Buffering is something that should be available in stm32duino or optionally by passing a *buf and a size of a virtual library which feeds the real com port. I can see that some projects may break if the serial is suddenly faster (non blocking) but the library would be better for having buffered serial. Even in the 90s we had buffered uarts 16550A made using RS232 much better and more reliable.

If you implemented both the old serial and say serialbuf would serialbuf be linked if it wasn’t used in the sketches code ? If so the user could just use serialbuf or serial. For new projects most people aren’t going to want serial to be blocking unless of course they are just going to wait for a reply to what they have sent.


martinayotte
Thu Jun 30, 2016 1:34 pm
BTW, for everyone, we should look that those RX/TX buffers are already implemented under F4.

The only thing is that USART_TX_BUF_SIZE/USART_RX_BUF_SIZE are hard coded to 256 characters.
It will be easy to add the new setTxBufSize() and setRxBufSize() functions to make them dynamically allocated.

For the blocking/non-blocking feature, it would be easy to do the same things that is already done in usb_serial, where fonctions enableBlockingTx() and disableBlockingTx() are present.

So, there is no needs to create a new class…


edogaldo
Thu Jun 30, 2016 1:58 pm
martinayotte wrote:For the blocking/non-blocking feature, it would be easy to do the same things that is already done in usb_serial, where fonctions enableBlockingTx() and disableBlockingTx() are present.

So, there is no needs to create a new class…


martinayotte
Thu Jun 30, 2016 2:16 pm
Of course, even having DMA+IRQ, it doesn’t prevent of having the blocking/non-blocking functionality.
Recently, I’d measured the throughput of usb_serial where the source was a file located on SdCard and discovered that non-blocking was defaulted.
viewtopic.php?f=3&t=1159#p14759
I think this piece of code on F4 is not even present under F1. (I think F4 is more advanced than F1 in those area)

edogaldo
Thu Jun 30, 2016 2:40 pm
martinayotte wrote:Of course, even having DMA+IRQ, it doesn’t prevent of having the blocking/non-blocking functionality.
Recently, I’d measured the throughput of usb_serial where the source was a file located on SdCard and discovered that non-blocking was defaulted.
viewtopic.php?f=3&t=1159#p14759
I think this piece of code on F4 is not even present under F1. (I think F4 is more advanced than F1 in those area)

simonf
Thu Jun 30, 2016 3:50 pm
edogaldo wrote:
so I’m not that sure it’s a very well tested feature…

martinayotte
Thu Jun 30, 2016 4:14 pm
For USBSerial, I’ve tested it two weeks ago (see above thread), and it is working as expected.

The call is NOT “this->usbEnableBlockingTx()”, so it not calling itself … :ugeek:
Don’t forget that the code is calling the C variant usbEnableBlockingTx() located in usbF4/usb.h, and therefore in usbF4/usb.c, it is calling the VCP_SetUSBTxBlocking() from usbF4/usb.c. (I know, it looks confusing, but it is LeafLab code … ;) )

Yes, it is only in USBSerial, I was just saying that it should be implemented also in HardwareSerial too so we can get both behaviour options.
And under F1, is not implemented too, not even on USBSerial, which means that it has a non-blocking behaviour, preventing send large amount of data without losing some bytes.


edogaldo
Thu Jun 30, 2016 4:42 pm
martinayotte wrote:For USBSerial, I’ve tested it two weeks ago (see above thread), and it is working as expected.

The call is NOT “this->usbEnableBlockingTx()”, so it not calling itself … :ugeek:
Don’t forget that the code is calling the C variant usbEnableBlockingTx() located in usbF4/usb.h, and therefore in usbF4/usb.c, it is calling the VCP_SetUSBTxBlocking() from usbF4/usb.c. (I know, it looks confusing, but it is LeafLab code … ;) )


martinayotte
Thu Jun 30, 2016 4:57 pm
edogaldo wrote:but “usbDisableBlockingTx()” is for sure broken: it will never toggle the “UsbTXBlock” flag off

edogaldo
Thu Jun 30, 2016 5:02 pm
martinayotte wrote:edogaldo wrote:but “usbDisableBlockingTx()” is for sure broken: it will never toggle the “UsbTXBlock” flag off

martinayotte
Thu Jun 30, 2016 5:59 pm
Right ! usbEnableBlockingTx() used twice !
I will send a PR to Roger … :)

EDIT : PR sent …


RogerClark
Thu Jun 30, 2016 11:48 pm
I’ve actioned that PR (for the F4)

martinayotte
Fri Jul 01, 2016 2:37 am
Thanks @Roger !
Thanks to @edogaldo for having find this untested bug !

edogaldo
Fri Jul 01, 2016 7:54 am
YW.
BTW, I just started learning something about using github and prs, next time I’ll try opening a pr myself.

Best, E.


RogerClark
Fri Jul 01, 2016 8:04 am
PR’s are the best for me, as they are easy to manage.

Though in this case I’d need to branch then merge the PR


edogaldo
Tue Jul 05, 2016 7:35 pm
Hi Roger, I would like to contribute to the Arduino_STM32 library; I applied following improvements to the F1 USART:

  • added buffered-interrupt based TX
  • buffers can be dynamically allocated via HardwareSerial.begin()
    • min RX buffer size: 16
    • default RX buffer size: SERIAL_RX_BUFFER_SIZE
    • min TX buffer size: 1
    • default TX buffer size: SERIAL_TX_BUFFER_SIZE
  • added HardwareSerial.enableBlockingTx()/disableBlockingTx() (blocking enabled by default)
  • improved ring_buffer to support the full buffer size instead of buffer size – 1

I’d like to send you the changes via PR but I’m a complete nebie to Github, could you please explain me the steps to submit a PR for your project?

Thanks and best, E.


martinayotte
Tue Jul 05, 2016 8:07 pm
@edogaldo,

First, you will need a github account and login into it.
Second, from Roger’s github, press “fork” link in the upper-right corner, it will create a fork from Roger’s repo to your account.
On your PC, do a “git clone <the_forked_repo_in_your_account>”
Edit/merge your changes there.
Commit them and push them into github, it will then appear into your account github web page with all commits history.
Now press “compare” in your github account fork, it will shows the diffs against Roger’s repo.
If you are satisfy with them, press “Create Pull Request” button, fill out the message to associate with the PR and submit.
Roger will then have to do the merge …


edogaldo
Tue Jul 05, 2016 8:53 pm
Thank you very much Martin, I could successfully create the PR.

Best, E.


RogerClark
Tue Jul 05, 2016 10:14 pm
@edogaldo

I’ve taken a look at the PR, but I have a worry about the change you made to Serial::begin() and a few other issues.

with
void HardwareSerial::begin(uint32 baud, uint16 tx_buf_size, uint16 rx_buf_size, uint8_t config)


martinayotte
Tue Jul 05, 2016 10:29 pm
RogerClark wrote:
I’m a concerned about the way the buffers are dynamically allocated using malloc.

RogerClark
Tue Jul 05, 2016 10:37 pm
Martin

I’m sure the core doesnt use new(), as if I use new() in a sketch the binary gets much much bigger as loads of extra libs have to be linked in.

These buffers would need to statically allocated

#define TX_BUF_SIZE 64
#define RX_BUF_SIZE 64

char rxBuf[RX_BUF_SIZE];
char txBuf[TX_BUF_SIZE];


RogerClark
Tue Jul 05, 2016 10:39 pm
On a pedantic note about PR’s

Please do not edit anything else apart from the change you are making

Do not tidy up any other code including removing any whitespaces or newlines, as Git treats all these changes as important and clutters up the difference displays that I have to wade though when looking at a PR

(Note I”m not the only one who has an issue with whitespace removal in PR’s Paul (Teensy) has posted several times about this to various places)


martinayotte
Tue Jul 05, 2016 11:09 pm
RogerClark wrote:
These buffers would need to statically allocated

simonf
Tue Jul 05, 2016 11:25 pm
martinayotte wrote:RogerClark wrote:
These buffers would need to statically allocated

edogaldo
Tue Jul 05, 2016 11:37 pm
Hi both, here my feedbacks:

  • about white spaces, tabs, etc..: I changed them for consistency anyway ok, got it, should I revert space changes?
  • about dynamic buffer size: I was for static buffer size (as in the standard Arduino UNO & DUE implementations) but I got from this forum it would have been a plus to have dynamic buffers and this implies dynamic allocation; my tests are fine on this
  • about the begin method:
    • arguments 2, 3 and 4 are optional: if you don’t specify them they are defaulted with the specified values
    • I left “config” as argument 4 (last) due to comment:
      /*
      * Roger Clark.
      * Note. The config parameter is not currently used. This is a work in progress.
      * Code needs to be written to set the config of the hardware serial control register in question.
      *
      */

RogerClark
Wed Jul 06, 2016 12:24 am
@edogaldo

I still think a PR is the best approach

I can pull to a new branch if you like, at the moment I don’t feel comfortable with pulling to either master or development

I think the consensus from another thread as that people wanted me to pull new enhancements to a separate branch first, so I think I better start doing this, even though it ends up with loads of branches

Re: Dynamic allocation.

I thought that people wanted to dynamically allocate buffers themselves and pass into the function

Re: comment about the config
Thats a really old comment, I can’t remember if it has been superseded. Either way, even if its not currently supported, I’d still prefer to add new optional parameters at the end of the function signature, so that when config is finally implemented, that the API will be as close as possible to the Arduino.cc API e.g. on AVR boards


martinayotte
Wed Jul 06, 2016 12:53 am
edogaldo wrote:
Of course if you think not worth the PR approach then let’s cancel it.

martinayotte
Wed Jul 06, 2016 12:55 am
RogerClark wrote:
Re: comment about the config
Thats a really old comment, I can’t remember if it has been superseded. Either way, even if its not currently supported, I’d still prefer to add new optional parameters at the end of the function signature, so that when config is finally implemented, that the API will be as close as possible to the Arduino.cc API e.g. on AVR boards

edogaldo
Wed Jul 06, 2016 8:55 am
Hi Roger, I got your concerns about dynamic allocation and (even if I’m on Martin’s side else I would not have used it), since I think missing buffered interrupt-based TX is a major lack in STM32_Arduino library, I’d like to find out a trade off solution to release that functionality.
On this purpose let me list my concerns respect to the solution you have in mind for dynamic buffer allocation (letting the users allocate custom buffers and pass them to the library):

  • this solution allows a compile-time level dynamic buffer allocation, not a run-time level dynamic allocation so, from a functional point of view it’s almost the same as defining the buffers sizes at pre-processor level and letting the user to change these defines
  • letting the user to allocate custom buffers does not prevent the need for the system to allocate default buffers for each device (even those not used) and respect to the pre-processor based solution is sub-optimal because leads to a situation in which, when the user allocates some custom buffer then the application will end-up having 2 buffers allocated for the same purpose but only one used (either the default one or the custom one) thus resulting in wasting memory

All said, do you think we can find a trade off solution to release at least buffered interrupt-based TX functionality?
What would you suggest?

Thanks and bye, E.


RogerClark
Wed Jul 06, 2016 11:34 am
We could just pass in statically allocated buffers, as well as the size of those buffers (but it needs 4 params instead of 2)

I think this resolves the problems of using malloc (which I think a lot of people will have an issue with)


edogaldo
Wed Jul 06, 2016 11:50 am
So no concerns about ram wasting due to buffers duplication?!

Pito
Wed Jul 06, 2016 1:50 pm
KISS: RX buffer 128 and TX buffer 128, both hardcoded.. :)

edogaldo
Fri Jul 08, 2016 4:49 pm
RogerClark wrote:We could just pass in statically allocated buffers, as well as the size of those buffers (but it needs 4 params instead of 2)

I think this resolves the problems of using malloc (which I think a lot of people will have an issue with)


Slammer
Sat Jul 09, 2016 8:25 am
#defines are good solution but not inside arduino IDE environment.
The IDE silently includes Arduino.h in the first line of the program (and this includes all other includes), so it is impossible to define the buffers in the program.
The other solution is to define the buffers with defines in command line arguments, but this requires changes in arduino IDE build system, or to have defined other variants with different buffer sizes.

edogaldo
Sat Jul 09, 2016 8:49 am
Slammer wrote:#defines are good solution but not inside arduino IDE environment.
The IDE silently includes Arduino.h in the first line of the program (and this includes all other includes), so it is impossible to define the buffers in the program.
The other solution is to define the buffers with defines in command line arguments, but this requires changes in arduino IDE build system, or to have defined other variants with different buffer sizes.

mrburnette
Sat Jul 09, 2016 12:48 pm
edogaldo wrote:
Yes slammer, you are right, I was trying to find a way to make it easy set defines but no success..

edogaldo
Sat Jul 09, 2016 4:50 pm
mrburnette wrote:
In the “old” days, one would write a separate GUI “conf utility” to fine tune the flatfiles used to actually specify the compile environment. Today, such a utility could exist in the /tools directory. In this way, it would be easy to set #defines and other configuration parameters.

Ray


Slammer
Sat Jul 09, 2016 6:54 pm
For those that compile arduino projects outside IDE (makefiles, eclipse, codeblocks, etc) it is very easy to make defines before inclusion of Arduino.h but it is impossible for IDE users.

mrburnette
Sat Jul 09, 2016 11:21 pm
Slammer wrote:For those that compile arduino projects outside IDE (makefiles, eclipse, codeblocks, etc) it is very easy to make defines before inclusion of Arduino.h but it is impossible for IDE users.

RogerClark
Sun Jul 10, 2016 3:21 am
I know there is a bit discussion of this, to try to determine the best way forward.

But just a few more things to throw into the mix

Unless I’m much mistaken we already have a RX buffer (ring buffer) in the uart class ???? (there is definitely code for this)

So, whatever the final solution is. We can’t change the existing default state of having a 64 byte Serial RX buffer

From usart.h
#ifndef USART_RX_BUF_SIZE
#define USART_RX_BUF_SIZE 64
#endif

/** USART device type */
typedef struct usart_dev {
usart_reg_map *regs; /**< Register map */
ring_buffer *rb; /**< RX ring buffer */
uint32 max_baud; /**< @brief Deprecated.
* Maximum baud rate. */
uint8 rx_buf[USART_RX_BUF_SIZE]; /**< @brief Deprecated.
* Actual RX buffer used by rb.
* This field will be removed in
* a future release. */
rcc_clk_id clk_id; /**< RCC clock information */
nvic_irq_num irq_num; /**< USART NVIC interrupt */
} usart_dev;


mrburnette
Sun Jul 10, 2016 1:22 pm
RogerClark wrote:
<…>
Other ideas worth considering, is adding very small TX buffering by default e.g. 8 or 16 bytes, and the let the user edit the #defines ??
<…>

mrburnette
Sun Jul 10, 2016 1:29 pm
The above being said, I am not opposed to a GUI (or command line) application being used to tune core parameters for STM32. A few radio buttons and a few checklists could control “tunable” parameters that would enhance peripherial features. If the utility was Java, it could be included in Arduino/tools and be easily accessible. Said utility would be STM32-centric.

Ray


simonf
Sun Jul 10, 2016 2:51 pm
Ok here is something radical you can use the same buffer for TX and RX. If the buffer was say 64 bytes long you could push rxed characters at the front of the queue and push the TX at the end of the buffer.

It would work perfectly if the TX speed was the same as the RX which I understand it is. you would just need to check that the RX buffer count + the TX buffer count was less than the Buffer size – 1. So you have 30 characters in the TX queue and 32 characters in the RX queue lets say another character is received you then have 33 chars in the RX queue. As they operate at the same speed the TX queue will drop by one char before the next RX character is received. Thus we need only one 64 char buffer for both TX and receive. The only thing that would be blocking is adding a TX char to the queue if the the total characters in both queues was 63.

This way we use no more memory than is already used in the current system and face it most people send a load of chars and then wait for a reply.

I may of not explained how this works very well I will try to put it in code to explain.


mrburnette
Sun Jul 10, 2016 5:44 pm
simonf wrote:
<…>
This way we use no more memory than is already used in the current system and face it most people send a load of chars and then wait for a reply.<…>

edogaldo
Mon Jul 11, 2016 3:03 pm
mrburnette wrote:
This does not mean that a new core cannot be developed radically departing from Arduino purity, but the original thinking of the old members was for “as-close-as-possible….”

mrburnette
Mon Jul 11, 2016 5:49 pm
edogaldo wrote:
@Ray, the main point of this thread is that Maple library does not provide buffered interrupt-based TX for the USART devices, which is ootb for Arduino boards (both UNO and DUE) and which makes USART TX for STM32F1 boards very slow (much more slow than in UNO boards) so, this improvement would be, imo, just the coverage of a lack in the libmaple implementation respect to the Arduino standard features.

Best, E.


edogaldo
Mon Jul 11, 2016 6:49 pm
mrburnette wrote:
Yea, I understand that.

But we have another responsibility to the real Maple folks since we completely inherited the Leaflabs codebase. We now carry that torch by default…
You can still read the docs. The forums will remain active until August 2016, but at that point they will be converted to a static archive. Consider checking out the resources and community at http://www.stm32duino.com instead.

Honestly, I do not know how many of our members own real Maple hardware and may be using STM32duino… but I see from time to time a large number of users that are not bots and are not logged-in. Anyway, it is something to think about. Hell, I don’t have any issue with changing F2, F3, or F4. It’s a small thing, but IMO worthy of some serious neuron activity.

Additionally, and this is a personal statement of fact, in 3 years I have not had a serial issue. Maybe I am just lucky or perhaps my code does not stress the uC significantly. I’m sure I could write something to force the issue to appear, but nothing I have done normally in 3 years has caused me concern.

All the above is why I abstained for making a big forum issue out of what I think, architecturally. Those that have been in this forum for a few years know that I’m not a shy member. I just think that members that have experienced issues or expect to experience issues have a voice. As for me, I just do anything I want anyway.

Ray


mrburnette
Mon Jul 11, 2016 7:03 pm
edogaldo wrote:
<…>
Yes, you carry the torch, but if you don’t go ahead with that torch, what do you carry it on for?
Why the F0, F2, F3, and F4 libraries can grow but the F1 cannot?
Come on, the name of this site is “Arduino for STM32” not “Libmaple forever” (by the way the original Libmaple is already protected in the libmaple repository available for posterity).
<…>

edogaldo
Mon Jul 11, 2016 9:14 pm
I’m not expecting anybody takes any side, I’m just trying to contribute to improve the library.
I opened a dedicated thread for my improvements: viewtopic.php?f=17&t=1235
For anybody interested I’d suggest to discuss details there.

Best, E.

ps: I replied to your questions in the other thread.
Pps: let me also add that original Maple boards are at the end F1 boards so they’d benefit for the improvements too..


Leave a Reply

Your email address will not be published. Required fields are marked *