Saving RAM used for PIN_MAP

Hi all,

I noticed recently that even though PIN_MAP is declared as const, it is stored in RAM.
As it is a constant, I don’t see why it should be stored in RAM, so I though it was the compiler or the linker messing with it for some weird reason.
I tried __FLASH__, but that gives me an error message saying it causes a type conflict with something else, so I decided to do a bit of research on that error message, and after finding some posts on how to force data in a certain section, I decided to try declaring it this way, to force it to go to .rodata which is part of flash:
extern const stm32_pin_info __attribute__ ((section (“.rodata”))) PIN_MAP[BOARD_NR_GPIO_PINS] = {

My RAM usage for the same sketch went from 9,432 to 8,888, flash usage went from 15,636 to 16,180, so I know it is indeed moving that to flash.
I know there were some old posts in the original thread trying to move PIN_MAP to ram, but I couldn’t find a conclusion to that.

I have a couple of questions for anyone that knows:
Is there any problem on moving PIN_MAP to flash?
As my maple mini is dead (I found the 1117 was fried, so I replaced it and the bootloader works, but not the sketches, I think the flash is faulty), if there is no known issue, can someone modify his board.cpp file to declare PIN_MAP that way and see if the sketches still work?

Thanks.

It will be most interesting to determine whether the change will impact performance. As the STM32F series is a Harvard architecture, you are moving the LU structure from one data access bus to the other (flash and SRAM on different bus.) Additionally, depending on how one reads the technical reference, there may be up to 2 clock delays associated with the flash access.

Flash memory

0-24 MHz – zero wait state
24-48 MHz – one wait state
48-72 MHz – two wait states
The prefetch buffer must be switched on/off only when SYSCLOCK is lower than 24 MHz.

I did a few Google queries and I cannot get a good feel as to which scenario is best: flash or SRAM.

Ray

I believe the initial intention may have been to put it in flash, as it was declared constant, and all other variables in that file have the __FLASH__ macro applied to them, except for this one, because doing so would fail to compile.
Is PIN_MAP used much? I did a quick search in the repo and didn’t see it poping up too often, I thought it may be used only when initially configuring a peripheral, but I may be wrong.

If any one decides to test it, may help to run some sketch intensive in IO, such as one of the SPI displays.

EDIT: I see it is used for digitalRead and digitalWrite. May be worth testing if someone has something intensive in PIN IO

Victor

I used a field in PIN MAP to store the mode of the pin (set by pinMode()) it will break some things if its in flash

But I agree it doesnt need to be in RAM

The mode of the pin can probably be read back from the hardware
I think I needed to do if for analogWrite or analogRead as on AVR you don’t need to set pinMode before calling either, but you did in libmaple

So I put code in to check the pinMode when (I think ) analogWrite is called.

In fact if we can get PIN MAP into flash it may fix the issue of PIN MAP not being initialised before code in global constructors is run, e.g. Hence issues with the OneWire library ( and other libs)

Edit.

Looking at the code, I think I must have decided that storing the pinMode was not worthwhile

I can’t any code that uses is

see /STM32F1/cores/maple/wirish_types.h

remove the last item in the struct “pinMode”

typedef struct stm32_pin_info {
gpio_dev *gpio_device; /**< Maple pin's GPIO device */
timer_dev *timer_device; /**< Pin's timer device, if any. */
const adc_dev *adc_device; /**< ADC device, if any. */
uint8 gpio_bit; /**< Pin's GPIO port bit. */
uint8 timer_channel; /**< Timer channel, or 0 if none. */
uint8 adc_channel; /**< Pin ADC channel, or ADCx if none. */
uint8 pinMode; /**< mode specific by pinMode call (Roger Clark added to optimize compatibility with Arduino API*/
} stm32_pin_info;

Roger,
I am a bit confused, PIN_MAP is already defined as a const in the current version of the code, so I wouldn’t expect that you modified any part of it during runtime.
The code should not even compile or could crash if you tried to modify it, right?

I see you are referring to fields in stm32_pin_info, which is not defined as const, but I was referring to PIN_MAP, which is in boards.h/boards.cpp, and currently defined as const, so unless casting it, the code should not even compile if you tried to modify it, right?

This is the line declaring it in STM32F1 / variants / maple_mini / board.cpp :
extern const stm32_pin_info PIN_MAP[BOARD_NR_GPIO_PINS] = {

Looks like if you tried at some point to check the pinMode before an analog write, you finally took that out, as currently it just sets the mode to PWM every time is called with no checks:

57 void analogWrite(uint8 pin, int duty_cycle8)
58 {
59 pinMode(pin,PWM);
60 pwmWrite(pin,duty_cycle8 * 257);// 257 maps 255 to 65535 (i.e 255*257 = 65535)
61 }

Also I can not find any function using or modifying that pinMode field.
I’ll try out later running a few sketches with that field removed and the PIN_MAP in flash, I should be getting a mini today from the post office.

Hi victor

I will take the pinMode variable out of the struct.

If we do need to store pin mode in the future to speed things up, we can put it in a separate array.

I.e. analogWrite will be a lot slower than it needs to be, because we have to call pinMode for backwards compatibility with the AVR boards, which don’t seem to need to set the pinMode in order to use PWM.
But in the STM32 there are extra pin modes, once of which is PWM.

I think this is something that probably needs to be documented, on this site somewhere

Perhaps we need a new section for STM32 specific programming.
Where we can put this stuff and also discuss the DMA and DAC etc

I found this post from the old Leaflabs forum regarding SRAM vs flash allocation:
http://forums.leaflabs.com/topic.php?id=992 Check out mbolivar response.

Ray

Ray,

Thanks for the link.

As everything in PIN MAP is const, it would be safer to declare as const, because the FLASH attribute is a linker directive, so i would be a bit concerned if only __FLASH__ was used, that the compiler would not know those values could not be changed.

The items in the stm32_info struct, are also all const, so it would be best if they were marked as suck, but more code needs to chance if we change that, because there are several functions which give errors if I change stm32_info members to const.

So as a first pass, if say, go though and change the const stuff, as its likely still to get put in RAM, and second passs (when the const stuff is working) would be to add the FLASH linker directive

It’s probably worth profiling the speed of digitalWrite before and after the change, as it looks up stuff in PIN MAP

Good reading Ray. Also some notes about speed later in that same thread, but as far as speed is involved, I think we could gain more from directly writing to the port registers when speed is needed, than keeping that whole table in RAM.

I need to test now that I have a couple of running Minis.

Victor

Thanks.

PS. Not sure if there is an example that will make this test easier, but if so, could you check things also work in an interrupt , as this seems to often be an area which doesnt work after we have changed this sort of thing

PS.

I’m around for a while on IRC

http://webchat.freenode.net/?randomnick … stm32duino

Roger,

I have been checking what happens with PIN_MAP, here some findings:

Whether PIN_MAP is in Flash or RAM, it is initialized with only the int values, all the pointers (GPIO, TIMER and ADC) are 0 initially.
I have verified that is the case no matter if I force it to flash or not. I don’t know at what point PIN_MAP gets populated with the correct values, I guess somewhere during initialization, but I would think having 0 values is what causes the other issues you pointed out with libraries trying to access IO ports. Perhaps they get the 00s rather than the correct values.

On a DUE there is a similar table, but has the correct values in flash. It only contains one pointer, and that pointer is to MCU register address, not to a strut, like we have in libmaple.

Changing it all to constants seems a bit more complicated than I thought and may require editing some functions, but I think is feasible, and I think that should solve the other issues with PIN_MAP.

Now, about being slower in flash, PIN_MAP gets accessed a lot for digitalRead and writes, but those function have a lot of other overhead too, so I am not sure they would be impacted that much, but I think that could be measured with some tests.

I don’t think leaflabs put it in RAM in purpose, but was a compromise for some reason. The set the ADC pointers as constant, but not the TIMER and GPIO ones, no idea why, except that perhaps they thought on having it all as constant and save RAM at some point, but did not finish that task.

Victor

Thanks for looking at this

I read something posted by BobC the other day to the IDE developers mailing list, which suggested that the Due had some form of initialisation work around to set this sort of thing up.

One think that could be tried is adding another array, in flash, but just put some hard coded numbers in it.

Then see if that gets data in it, in the FLASH .

If so, perhaps we can slowly work out why PIN MAP is zero if its in FLASH

Re speed loss if PIN MAP is in flash

I think the benefit of pre initialised data, would outweigh any speed losses, and as you say, there are speed improvements we could do to the API functions e.g. pinMode

Anyway

Thanks again

Roger

Well, I am about to give up on the PIN_MAP going to RAM. I think it is a compiler or linker issue.

I have been testing and if any of the date in the PIN_MAP struct is a pointer to another constant or variable, then it won’t populate the value at compile time, it will leave it as 0.
If I do like the DUE does, that there is a pointer to a memory address defined in a define line, that populates there correctly at compile time.
I have changed the all the pointers to *const and still have that issue.
In fact I see that TIMER1, GPIOA GPIOB, etc, go to flash addresses, so the compiler is correctly taking those as constants, but those addresses do not show up in PIN_MAP. My guess is that something is populating that data at runtime, so when I force it to flash, those stay with a value of 0 and nothing works.

I have tested all this in a computer with only the maple ide and codesourcery gcc 4.4, I will repeat the test in a computer with the arduino IDE and GCC 4.8 or even 4.9, but I also found this old post on weird issues also related to placing pointers in a struct in ROM, only I don’t see how to apply his solution to our case:
http://stackoverflow.com/questions/4962 … arm-device

Victor

Thanks for looking at this

If its a linker settings issue, its going to be a bit of a steep learning curve, for us all I think, because I looked at the linker scripts a while ago, when we were moving the code start address for Bootloader 2.0 and the linker scripts use their own language which looks like nothing I’ve come across before ;-(

I still think its worth making a second array e.g.

PIN_MAP_FLASH and start by just putting numbers in it and see if that gets put in flash

Btw. How do you test if its in flash or not?

Does Serial still work when you mess around with this stuff ?

Thanks

Roger

Any clues in the disassembly listing of Blink.ino?

I did a disassembly w/ source which is attached as an artifact.

Code compiled and disassembled:
/*
Blink: Turns on the built-in LED on for one second, then off for one second, repeatedly.
Arduino 1.7.2 Maple Mini Bootloader 2
Sketch uses 15,276 bytes (12%) of program storage space. Maximum is 122,880 bytes.
Global variables use 4,504 bytes of dynamic memory.
Ported to Maple from the Arduino example 27 May 2011 By Marti Bolivar
*/

#define BOARD_LED_PIN PB1 // Maple Mini board LED pin# 33
#define ONE_SECOND 1000 // 1000 mS
void setup() {
// Set up the built-in LED pin as an output:
pinMode(BOARD_LED_PIN, OUTPUT);
}

void loop() {
digitalWrite(BOARD_LED_PIN,!digitalRead(BOARD_LED_PIN));// Turn the LED from off to on, or on to off
delay(ONE_SECOND); // Wait for 1 second (1000 milliseconds)
}

Ray,

There looks like there are small bits of static data, but the pin map looks like a set of move instructions e.g. moving the data from rom to ram

objdump -CS will demangle the C++ names

Roger, I had verified that the int8 values were there correctly, it was the pointers to TIMER, ADC and GPIO that were missing, even though those pointers themselves were saved in Flash, so they were at a very static position, but still would not show up in PIN_MAP, only 00000000 for those fields.

Looking at libmaple and due, the due makes something like this (not real values, just the idea)

#define GPIOA (gpio_dev *const)0x800000

Them in the pin array, it uses GPIOA.
In libmaple instead, we have a gpioa for type gpio_dev, then we have a pointer gpio_dev *const GPIOA = &gpioa

And the pin_map contaions GPIOA.
So even though GPIOA itself was taking the right value in flash and pointing to the RAM position of gpioa, the linker for some stupid reason would not put that address there in pin_map.

I verified that I put an integer works, if I put a value defined in a define, works, but if I put a pointer to something, it will not work (I even created new poiners and gave them a certain value in hex, still would not work).

But… I found a solution. If I populate the table with the address that GPIOA contains (&gpioa), that works flawless.
Now, gpioa (in lowercase) was declared static, so I had to modify the timer.h, timer.c, gpio.h, gpio.c, adc.h and adc.c to declare the variables extern rather than Static.

I have no clue why leaflabs declared them static, given that right next they declared an extern pointer to each of those statics, so they could be modified from anywhere in the code, and in fact they need to be modified to manage the pins anyway.

I have compiled the blink sketch and uploaded it, and it is working. I have alse check the position of pin_map in the .map file, and checked the bin file and has all the right values in the right places.

So at this point pin_map can be stored RAM, but requires modifying a bunch of different files and making the timerX, adcX and gpioX variables externs. Still they were taking memory and they were accesses thru the pointers, so I don’t see a disadvantage in making then extern, but if you think otherwise let me know.

If there is any good example to test the issue that you had with classes manipulating the pins, let me know, I don’t think I remember many details on that, but could be related to the fact that the pin_map table was not complete until run time, and now it is complete at compile time, so we can test.

I’ll create a new repo to upload the modified files, as I made a copy in my HDD to not mess up my working install (not that I haven’t messed it up already…)

Furthermore, this is the blink sketch memory usage with PIN_MAP in FLASH, and in RAM (I use libstd nano, otherwise ram would go up to 4xxx):
Sketch uses 13,044 bytes (10%) of program storage space. Maximum is 122,880 bytes.
Global variables use 1,872 bytes of dynamic memory.

Sketch uses 12,744 bytes (10%) of program storage space. Maximum is 122,880 bytes.
Global variables use 2,432 bytes of dynamic memory.

BTW, there is no need to disassemble to see where PIN_MAP goes, our liker options are set to create the .map file automatically, which shows the address for the variables. I.e when PIN_MAP is in RAM:
.data.PIN_MAP 0x20000068 0x220 C:\Users\Victor\AppData\Local\Temp\build9195735699750058409.tmp\board.cpp.o
0x20000068 PIN_MAP

When in flash:
.rodata.PIN_MAP
0x08004ea4 0x220 C:\Users\Victor\AppData\Local\Temp\build9195735699750058409.tmp\board.cpp.o
0x08004ea4 PIN_MAP

thanks rick

Can you send me the updated files or do a pull request

Thanks

Roger

I am still in an older revision of the repo, with IDE 1.6.0, so I still have toggle led.

I did a test of a for loop toggling the led for 1 million cycles, and with the PIN_MAP in ram it takes 640ms to complete the loop. With pin_map in Flash 738ms, so about 100ms more, that results in 15% more time to complete that loop.
Given that this loop only does pin manipulation, with no other overhead, I would say it has less overhead than more routines.

This is the sketch for reference:

void setup() {
// Set up the built-in LED pin as an output:
pinMode(BOARD_LED_PIN, OUTPUT);
Serial.begin();
delay (5000);
Serial.println("Tests about to start");
}

void loop() {
Serial.println("Starting digitalWrite test");
uint32 t = millis ();
for (uint32 n=1000000; n>0;n--){
toggleLED(); // Turn the LED from off to on, or on to off
}
t = millis()-t;
Serial.print(t);
Serial.println(" ms. for 1.000.000 cycles");
toggleLED(); // Turn the LED from off to on, or on to off
delay(1000); // Wait for 1 second (1000 milliseconds)
}


Victor

The other big benefit of having the PIN_MAP in FLASH is that its a solution to the issue of the PIN_MAP not being initialised before the global constructors

I know we may be able to fix the global constructors in a different way, e.g. find another way to init the PIN_MAP in ram before the global constructors, but it does seem to be a good solution in the mean time, and I think it would be easier to tell the compiler its in RAM than it has been to move it to FLASH

Roger, I just created a new repo to put my modified files. I did not want to send a pull request to your repo because I am not in sync with you and don’t want to mess something up in your or mine, so I just created a new repo with the STM32F1 folder.
These are the files I had to edit to make it compile to Flash. After doing the changes to the pointer in pin_map, I don’t even need to tell the linker to put it in flash with an attribute or anything, it does it by itself
STM32F1_Modified\cores\maple\libmaple\adc_f1.c
STM32F1_Modified\system\libmaple\stm32f1\include\series\adc.h
STM32F1_Modified\cores\maple\libmaple\gpio_f1.c
STM32F1_Modified\system\libmaple\stm32f1\include\series\gpio.h
STM32F1_Modified\cores\maple\libmaple\timer.c
STM32F1_Modified\system\libmaple\include\libmaple\timer.h
STM32F1_Modified\system\libmaple\stm32f1\include\series\timer.h
STM32F1_Modified\variants\maple_mini\board.cpp

The modified repo is here:
https://github.com/victorpv/STM32F1_Modified/

Would be nice if a couple of people test it and make sure I didn’t break anything else.
I may have modified some other files in the process, but I am almost certain those listed above are the only ones needed, so you don’t need to download the whole thing if you don’t want to.

Hi Victor

I can create a branch with this in it, and people can select that branch if they want

Thanks

Roger

That could be a good solution.
I am still uploading files to that modified repo.

EDIT: I think it is all up. As long as the files listed above are up, that should be all that’s needed.

Victor,

I manually copied the individual files from that repo into a branch of my repo
(see https://github.com/rogerclarkmelbourne/ … P-in-FLASH )

I had to make one minor change to board.cpp as yours was a little out of date, but the other files were OK.

I updated board.cpp in the Maple mini and also the generic STM32F103C series, as these seem to be the boards that are used by most people

I will write an announcement and hopefully @ahull and Rick etc can test on their linux systems, not that it should make any difference

GPIO seems to work OK, and Serial is still working

I have an SD card connected, and amazingly that seems to work ! I’m amazed as it never seems to work at the best of times !

Anyway.

I will also test OneWire in a minute to see if we can use the generic version of the library (That Paul still hasnt actioned the pull request for !!)

Oops

Looks like I didnt do all the stuff in the 103C

Yeah my repo is a bit outdated, that’s why I didn’t want to send a pull request to yours and end up reverting the latest changes you did.
I need to update to yours, but I want to finish with the SPI dma functions first, so if something fails I know what changes I have made.
Once that is ready, I’ll update to yours, then add the SPIdma functions again, test, and send a pull request for those.

I’ll open a different thread about the SPI dma. It is nice to have different threads for everything now

EDIT: Is there any quick sketch I can run to see if the global constructors pin manipulation works?

Just modify any library and add pinMode into a constructor.

However, this isn’t a guarantee that all work ok, because this was always an intermittent problem ( see the stuff about the undefined order in which globals are setup and initialises , which I wrote in the original post)