Battling combinatorial explosion

jcw
Thu Oct 29, 2015 3:13 pm
The number of board / chip / speed / memory size / USB presence + enable pin / memory offset combinations is growing rapidly. Leading to a fair amount of duplication of code in all the “variants” sub-folders.

Maybe it’s possible to simplify this?

Perhaps as follows: add a board type called “Custom”, which looks at say a “boardinfo.txt” file in the sketch folder, and then generates various settings, include files even, before calling the toolchain and upload tools. This could then cover all the generic F0/F1/F3/F4/F7 boards out there, now and in the future. Such an optional per-project boardinfo.txt file is basically a way to “inject” config settings and code/linker definitions into the tasks launched from the IDE.

Such on-the-fly generation of files will have to work on Windows, Mac OSX, and Linux, but there are probably several suitable shells or scripting tools to pick from.

I haven’t thought this through, but am willing to contribute and implement/test things, at least for Mac OSX.
Just kicking off the idea here, to see how useful & viable it would be.


Rick Kimball
Thu Oct 29, 2015 5:47 pm
I don’t think we should encourage a wider variety of boards. I think the Arduino succeeded ( as well as Apple ) by limiting the choices and getting a larger body of applications that are known to work without screwing around testing an infinite number of platforms.

.. my 2 cents.

-rick


mrburnette
Thu Oct 29, 2015 5:52 pm
jcw wrote:
<…>
Perhaps as follows: add a board type called “Custom”, which looks at say a “boardinfo.txt” file in the sketch folder, and then generates various settings, include files even, before calling the toolchain and upload tools. This could then cover all the generic F0/F1/F3/F4/F7 boards out there, now and in the future. Such an optional per-project boardinfo.txt file is basically a way to “inject” config settings and code/linker definitions into the tasks launched from the IDE.

Such on-the-fly generation of files will have to work on Windows, Mac OSX, and Linux, but there are probably several suitable shells or scripting tools to pick from.
<…>


RogerClark
Thu Oct 29, 2015 8:09 pm
@jcw

Yes. I noticed this as well.

I wondered if the Boards Manager would be a solution.
( or equivalent using zip download of cores)

i.e split each series into a separate board, e.g. F1T, F1C. etc

My Boards menu is now incredibly cluttered as I have Avr,Sam, Stm32, Esp8266 and now nRF51822

The problem at the moment is that if I split into separate git repos and try to use git submodules, the zip download from github does not populate the submodules :-(

However a solution is to write scrips and use the github releases mechanism

But, as ever I have not had time to investigate this option

BTW. i also have the stm32duino github account, so if we want to have a clean slate and use separate repos or the releases mechanism, we could use that account instead of my personal account ( as my account also contains a lot of other rubbish I occasionally work on)


RogerClark
Thu Oct 29, 2015 8:18 pm
BTW.

This is also a problem with the bootloader.

The number of make targets is getting silly.

I did consider trying to compile the bootloader online. I think my personal hosting ( where this site resides) is on HostGator, and I have ssh telnet access and I think I used to be able to run cgi binaries i.e which could be the ARM compiler.

So it may be possible to have a web page where you select options and then it compiles a custom bootloader which you then download.
But to save server load, Id also need to cache existing bootloader variants.

… As you can tell, I do think about this stuff a lot, I just need an army of minions to carry out my mad plans ;-)


jcw
Thu Oct 29, 2015 9:17 pm
Another way to simplify the menu structure, could be to have one entry for F103’s (or perhaps one for medium, high, and XL each), and then put all the package and memory size combinations in the variants menu.

Or F1, F3, F4 board menu entries, and perhaps one Maple, one Nucleo, etc. for mainstream product lines.

It really depends on where you want to take this.

If the goal is simplicity, then either very few entries ought to be supported at all, or a more elaborate approach to handling this combinatorial minefield will at some point be needed, IMO. Or as you suggested, partial download sets, but these may hit the same issues again a bit further down the road.


mrburnette
Thu Oct 29, 2015 9:27 pm
RogerClark wrote:<….>
… As you can tell, I do think about this stuff a lot, I just need an army of minions to carry out my mad plans ;-)

RogerClark
Fri Oct 30, 2015 12:51 am
Thanks Ray….

:lol: :lol: :lol:

I knew that was coming as soon as I’d pressed send ;-)


sheepdoll
Fri Oct 30, 2015 2:11 am
I think I answered this thread in the one about ChibiOS HAL.

To recap, I use Git to create a branch for each variant that CubeMX sets up and generates the HAL code. By enabling the branch for the core currently under use, switch to that branch. This way the Arduino app only sees a few cores that are useful, rather than the whole world. This does however entail having open and stashed branches that are not always pushed back to the root. On the other hand GIT is designed for multiple artist working at the same time on different parts of a large project.
CubeMX also seems to be designed to work this way. When the *.ioc is logged in git, there is a full revision history of it. If one adds the User or glue code inside the comments in the HAL .c project, When the Cube regenerates the code, the existing user code remains intact.
It also seems like magic. Enable a F0 variant branch and Arduino only sees F0 in the menus. Move back up the tree and enable a F4 branch and the API sees F4 in the menu. It really does seem like magic, and show the true power of source control.


RogerClark
Fri Oct 30, 2015 2:32 am
@sheepdoll

Unfortunately I think only a small proportion of users of the core, use git.

From what I’ve read most people just download the zip file


jcw
Fri Oct 30, 2015 8:25 am
Enable a F0 variant branch and Arduino only sees F0 in the menus.
Do you have to restart the Arduino IDE in between?

jcw
Fri Oct 30, 2015 8:54 am
To recap, I use Git to create a branch for each variant that CubeMX sets up and generates the HAL code.

If that’s the audience at which Arduino-STM32 is targeted, then great. Sounds like a flexible and powerful setup.


sheepdoll
Fri Oct 30, 2015 5:45 pm
jcw wrote:Enable a F0 variant branch and Arduino only sees F0 in the menus.
Do you have to restart the Arduino IDE in between?

jcw
Fri Oct 30, 2015 6:26 pm
Yes, too bad.

I’ve been thinking a bit more about your git-branch approach, I use the “GitHub for Mac” app quite a bit, which makes it very easy to pull code from that site, but also to switch branches, check in changes, and even submit pull requests. There’s also a “GitHub for Windows” app. On Linux, I assume everyone will be familiar with the command line, so no issues there.

https://desktop.github.com

Maybe there’s a not-too-complex-or-brittle way to describe the setup process for newcomers. The advantage being that this mechanism exists today, once you know how to switch branches. Something like: download/install the IDE, download/install GitHub, checkout a repo in the right place, restart the IDE.

I don’t know how easy it is to maintain an entire set of branches. What if you want to apply a change which needs to carry through in multiple, or even all, the branches? Use two separate repositories next to each other, perhaps? One single-branch with common stuff, and one multi-branch with references to the common stuff?


jcw
Fri Oct 30, 2015 7:02 pm
Another thought – combining the ideas of Git and ChibiOS: how about managing it all through symbolic links?

RogerClark
Fri Oct 30, 2015 8:24 pm
Im not sure whether symbolic links will work on Windows??

mrburnette
Sat Oct 31, 2015 12:12 am
RogerClark wrote:Im not sure whether symbolic links will work on Windows??

Leave a Reply

Your email address will not be published. Required fields are marked *