USB Composite library question

I’ve got a question for users of my USB Composite library. Background to the question: Currently, the library works like many Arduino libraries, namely it declares a number of handy objects like Joystick, USBHID, and you can just use the object directly. However, a side-effect of this is that a bunch of code you don’t need is linked in. E.g., even if you don’t use the serial port, the CompositeSerial object gets linked in. In theory, a smart compiler might be able to figure out how to leave out the objects, but the ARM version of gcc is not smart enough (I even tried the latest gcc 7).

I could modify the library so you have to declare your objects before using them. E.g.:

USBHIDDevice USBHID;
HIDKeyboard Keyboard(USBHID);
HIDJoystick Joystick(USBHID);
USBCompositeSerial CompositeSerial;

This makes the library a bit more cumbersome to use, but would save both flash and RAM. It could save almost 1kb of RAM in some configurations (e.g., if you don’t use CompositeSerial).

Should I make the change? I would like feedback.

I vote for yes.

thanks, i’ve not yet use the library literally, but yes separating them would be good.
i’m trying to get on the HID bandwagon but i found the protocol complex to understand. Unlike some of the simpler usb protocols HID use very structured set of commands. Often comprising as sequence of many commands (HID Report Descriptors https://eleccelerator.com/tutorial-abou … scriptors/) to send individual keystrokes or mouse movements. The main usb HID class specs available on http://www.usb.org/developers/hidpage/ isn’t too clear about how the sequence of HID Report Descriptors commands should be built for individual use cases as well.

however, i happened to run into this article about turning the pro-micro into a USB keyboard
https://www.sparkfun.com/tutorials/337
i’d think we can implement similar API so that code like
Keyboard.write('z'); // send a 'z' to the computer via Keyboard HID

I agree

Don’t instantiate an instance of the library from inside the library.

+1 for letting clients declare the objects they need.

Cheers, E.

also +1 and again a big thank you for this valuable contribution (this library)

[ag123 – Wed Aug 15, 2018 6:10 am] –
i’d think we can implement similar API so that code like
Keyboard.write('z'); // send a 'z' to the computer via Keyboard HID

Done as of 0.90. This will require minor rewrites to all code that uses the library. I’ve revised all the examples to give guidance and added a brief upgrading guide file.

While I have absolutely no experience of using your library, I would agree to not instantiating an instance of an object within the library that defines it.

I have had numerous … discussions … with OO programmer who want to build systems where EVERY object and meathod is created at signon. This is a horrendous waste of CPU time and resources. Thought needs to be given to what SHOULD be included in a library and what can be left out.

Whopper

[Whopper – Wed Aug 22, 2018 1:52 pm] –
<•••>
I have had numerous … discussions … with OO programmer who want to build systems where EVERY object and meathod is created at signon. This is a horrendous waste of CPU time and resources. Thought needs to be given to what SHOULD be included in a library and what can be left out.
<•••>

There is a disconnect between OOP PC/minicomputer programmers and embedded system programmers. With gigabytes of 64-bit RAM and an OS with a decent VM manager, there is no reason to not instantiate objects before use; it also sets the application memory footprint early so the OS can efficiently manage the app.

Libraries in Arduino account to a large degree for the popularity of the Arduino-ecosystem. Frankly, many weak programmers simply could not interface with complex sensors without libraries. Libraries and h/w use examples get a novice programmer “up and running” quickly. Adafruit and Sparkfun also understand that sensor libraries sell over-priced hardware; a portion of the over-pricing going to online forums and chat support.

Many libraries would be better written as straight use functions. Often the need for OOP is simply to encapsulate the library … something better done in some cases with namespace.

Ray

+1 for Ray, bare metal is *different*, it used to be simply registers, addresses and simply assembly language, these days c++ & libraries seemed to make bare metal ‘high level’, and one would soon find out the truth that blue pill / maple mini has 20k sram, that’s all and every thing needs to live in the 20k world

i’m curiously suspecting if one day stm may after all release a ‘monster’ (in terms of the physical foot print) stm32 with >1 MB sram, the gap between mcu and ‘application’ processors is pulling ever closer as time goes by

deeper below bare metal, i discovered a ‘forgotten’ *natural* world that’s made up of analog voltages, currents, capacitances, resistances and op amps, these days analog & digital is pulling ever closer and arduino, mcus is pretty symbolic of it

[ag123 – Wed Aug 22, 2018 6:01 pm] –
<•••>
deeper below bare metal, i discovered a ‘forgotten’ *natural* world that’s made up of analog voltages, currents, capacitances, resistances and op amps, these days analog & digital is pulling ever closer and arduino, mcus is pretty symbolic of it

One gets a feel for the power of Verilog with the Cypress PSOC series. Once the frustration give way to the insight of the internal building-block analog/digital network fabric, a whole world of possibilities open up. My short time spent with the PSoC was more than interesting.

Now, new fun:
FPGA Arduino

Ray

Speaking as someone who has spent years helping others find glitches in their code, hardware, and once in a while their basic approach to the problem.
You always want to make a library as specific as possible.
Done correctly, library trimming saves time, hassle, and memory.
This is an embedded processor, it will never be fast, but it can be fast enough.

When I look at a library for embedded systems, I tend to compare it to dos device drivers.
I need to be able to initialize/allocate, configure, get/set status, send, and receive.
While a full blown PC can afford more than that, an embedded system can not.
Program for the cases you are worried about, and bit bucket the rest.
Do so deliberately, so it never happens by accident.
If it happens by accident, you have no place to start looking for trouble except bulk tracing.
If you dispose of deliberately, then you only need check your dispatcher.
It also creates hooks that are useful for expanding functions later.
Adding if/then/else or select/case statements is quick and easy if structure your program cleanly.

Allocate on initialization, not at boot, is my rule.
That can mean it still happens at boot, but that it does not have to.
So if a jumper disables something, you can ignore it after checking the jumper value.
People would be shocked at the memory/performance lost because of initializing disabled hardware, or checking status on things that are not being used.

Neither use nor address any hardware you do not have to.
Neither use nor address any memory you do not have to.
Virtualization is a memory and performance hog, try to avoid it.
A library should clearly state prerequisites, and in embedded systems, should simply incorporate it instead of just including it.

I look forward to seeing a trimmed usb library.

Janus.

To take Janus’s post further to the darkside…

It is my belief that Arduino Libraries are for the Quick & Dirty, get up and running quickly crowd. Unless one really analyzes the library, one can never be entirely sure how/why it works as it does. Outside of the core libraries, I generally like to encapsulate the library .h and .cpp files directly into the sketch. This does two things:
– It allows you to trim and customize the library to your needs
– It encapsulates the library from future published updates

The Arduino IDE tab-metaphor makes incorporating libraries easy. The .h file is on one tab, the .cpp file is in another tab. Yes, Arduino’s craziness about how code is compiled and link can cause you to fail, but usually some editing in the .h file will solve variable scope problems… the biggest issue is automatic function prototyping which in my opinion is Arduino’s worst implemented feature… a curse. Code reuse is great, if it is your own code. Libraries are great if you dig through them to see how they work. Sometimes, you only need a small part of a library’s code, and while the GCC smart compiler is reasonably good at tossing out non-used modules, it does not always make the correct decisions.

Arduino was implemented to allow the sketch author to worry about the sketch and have the libraries do magic without peaking into the lib. This paradigm will make you a far worst programmer. Reminds me of the state of medicine: all the doctors are specialists and when something goes wrong there are fingers of blame pointing in all directions.

I remember when I was a kid and my Mom would scold me for eating my meal one item at a time… she would say, “Do not just eat one thing at a time – it is all going to the same place.” Libraries are like that, once the modules are compiled, they are all liked into the binary and gets downloaded as a blob.

Get to know your libraries, especially the ones used often. Pull the libraries into your sketch source directory and edit them as you need: cut out clutter, change buffer sizes, create new methods for classes. If 80% of your code is some other author’s libraries, is your 20% really the main program?

Ray

Encapsulate libraries, this was one thing I really loved on UECIDE while I worked with PIC32 _> https://uecide.org/
If you insert any library, UECIDE will do an automatic copy to your folder. On the left side you can even choose the *.h and *.cpp file for editing.
Sadly it was not possible to work with the STM32duino core on this IDE.

It wasn’t my intention to place myself on the sithdex, but hey, whatever works.

The Jedi in me is happy to see the arduino ide and support system.
It makes it easy to get into embedded stuff, and get some results you can see.
The community is supportive, the hardware affordable enough for nearly anyone to experiment with freely.
It is bright and hopeful.
Quick and dirty at least provides a {an affordable} starting point, which is more than most so called professional software/hardware typically does.

The Sith in me, is laughing.
While the arduino ide has the strengths of its virtues, it also has their weaknesses.
Advanced learning requires compare and contrast, which requires isolation.
Because the arduino ide uses your user profile, it makes it impossible keep isolated copies.
Which is why I recompiled it after altering my copy to be able to keep everything in the portable directory if present.
My personal fork no longer makes an entry in my user profile if I make the portable directory first.
The change was surprisingly small.

The second problem is that the arduino ide does not compile files, it compiles directories.
Which means it compiles every file in your sketch directory, as well the board support directory, and any library directories as well.
Not being a real java programmer, I am debating how to go about fixing this issue.
To me, this issue is a mosquito sized problem, one that hunts alligators for decent meals.
This means that code you never see in the ide can be compiled into your program.

The third problem is there is no standard uninstall system to remove the stuff you no longer want.
Without house keeping, detritus will accumulate, which is bad, or you will spend increasing amounts of time on startup.
Which is why I made mine truly portable.
Now I can get it where I want to start, then make a copy per project, with my baseline safely tucked away.
Which lets me incorporate any changes I want to keep, without adding to my startup time on new projects.

The Bendue in me however, has a different view.
The arduino ide is not intended for professional use.
It is intended as an introduction, not a commercial product.
Constantly starting over is not only anticipated, but planned on.
You are supposed to start over from scratch over and over as you learn.
The best way for those with knowledge to advance theirs, is to pass on that knowledge, to help others learn.
This enables you see your knowledge through another’s eyes.
Which is why you learn as much, though very different lessons, from teaching, as you did studying.

Janus.

Is the portable version not officially done?
https://www.arduino.cc/en/Guide/PortableIDE
But thanks leading me to this, putting everything into the same user folder drives me nuts.
So I can setup separate IDEs: One for AVR, one for STM32 and one for ESP

The portable provided by arduino still uses your user profile.

The sourcecode for the arduino ide is littered with these.

//folder = new File(System.getProperty(“user.home”), “sketchbook”);

Since this is used outside the area where the portable directory support was added, many files still reside in your user profile.
For my own part, I do not care if the non portable version still works properly after my mod.
There are very few pieces of software I use that are not setup to be portable.

Changing user.home to user.dir uses the launch directory instead of your profile.
Base, sshconfigfilesetup, setup & setup, all *.java files, were all that needed to be changed.
If you use windows, grab ztree, log the source tree, select all, then search for ‘user.home’, to see for yourself.

Janus.

The arduino ide is not intended for professional use.
It is intended as an introduction, not a commercial product.
Constantly starting over is not only anticipated, but planned on.
You are supposed to start over from scratch over and over as you learn.

I do not agree with that premise in total. Agreed that the Arduino IDE is not a professional product, but it is adequate for newbies and even some talented old-timers. The IDE does not always make programming easy, but neither does it hinder one too much. It’s just what it is.

The concept of starting over by design is not completely accurate. The user can always do a File/Save As. This allows one to have multiple templates – open one and save it as your new starting code. As the IDE can have multiple files open, cut & paste works well. The tabbed metaphor actually works very well and while it was designed for multiple .INO files it works for library encapsulation.

After all, the Arduino IDE is free. Whether there are better free IDE’s is not the issue, the issue is simply that for a majority of users, the IDE works acceptably. This means that forum members in the many “arduino-ish” forums have a common-denominator for discussion and teaching.

Ray

My bad, I was unclear on what I meant.

I mean starting over with each hardware project.
I make it a point to start with a clean copy of whatever environment I am using for each project.
The design of the arduino system seems designed to encourage this.
Making each iteration more compact and precise.

To me, the fact that the arduino is not a ‘professional’ package is a strength.
For what it does, it is actually simply structured.
Most of the things I disagree with about it, can be alleviated.
It also lacks feature fluff, which is a very nice bonus.
Everything it has, it has for a reason.

Much better than many systems I have been forced to use.
It is also easier to use than ladder programming, and if that makes no sense to you, feel glad.

Janus.

on a different note, this usb composite library is a volunteered effort, hence, i really thank apruss for investing effort in this and sharing this

i think separating the use cases / device classes has a rather strong merit, that would make usb-serial on its own, usb-hid on its own, usb-storage on its own etc. within usb-hid i think there could be even finer sub-division of the ‘use cases’ i’m finding that usb-hid is literally ‘pretty big’. if one make say a keyboard + joystick + mouse + sensor feedbacks (e.g. temperature, pressure, etc) i think hid could literally cater for it.

making a usb-composite device would then take a little more effort, perhaps a little bigger or if one needs to fork it and create a ‘custom’ usb-composite version that is size optimised for the multiple purposes.

the notion of developing distinct use cases / device classes on their own is that there are a large multitude of usb device classes / use cases
http://www.usb.org/developers/docs/devclass_docs/

this being a volunteer effort, others could then chip in and develop other device classes e.g. usb-audio, usb-imaging, usb-ethernet, usb-irda in a similar framework and extend this library.
that would make the BP/MM live up to what it really is: a generic usb device, its role dependent on how one makes the sketch and this library that runs on it and perhaps any additional adjacent peripherals

Agreed.

Most inhouse projects start small, for one job, then someone starts just adding things at random.
When I have had to restructure existing code for customers, I always use the same basic structure.
Device driver, usb_base.h as an example, I am not very original at naming things.
Then do something with it, usb_hid.h to follow the theme here.
Expand that with usb_keyb.h, usb_mouse.h, usb_gamepad.h, or more of the like, each of which could #include <usb_base.h>
I am not a real C/C++ programmer, so excuse me if I get some of the syntax less than perfect.

That leaves usb_raw.h if you want the bare packets for some reason.
usb_temp.h for a temperature sensor.
usb_bt.h for bluetooth, which a bluetooth responsive program could include.
usb_???.h for something no one here has thought of yet.

Chain them backwards via includes, and only the pieces needed are ever included.
Thus usb_hid.h just sends and receives hid packets, but nothing else.
Jut as each layer only does its job, nothing else.
You don’t need to plan for every possible value, just what you will be using.

This is what I had planned on doing with usb, once I actually managed to figured out anything about it.
This library will save me a lot of time, and anything I can improve about it, I will post back here so if it helps others, they can use it as well.

Janus.

one thing to note that writing a usb device class driver (e.g. hid) library is pretty much the app / sketch itself. the main arduino loop() does ‘nothing’
hence, i’d encourage ‘observers’ to try it out and possibly extend it, i.e. write new usb device class cases which could be added to the ‘library’.
this ‘library’ is literally a collection of apps / sketches. it isn’t really simply a ‘library’

Before people write implementations of new usb classes, I would prefer to refactor the code. It currently has one major limit: a composite device can only have one instance of each plugin (e.g., only one serial).

For my part, I am not even starting until I finish getting F107 board support working.
So far, that is lots of fun already.

I am going to wait until someone makes sense on how the packets are received or transmitted before I do anything.
Right now, usb is a mess, and I fail to understand the flow logic.
Thus I will wait until I manage to find a demo or program that uses usb otg, then disassemble the result to analyze on the assembly level.

I know people will say to just read the sourcecode, but I am not a real C/C++ programmer.
What I do is match existing patterns until I can see under them.
The logic level is where I work, and C/C++ is tedious at best in my eyes.

I don’t really like the arduino ide much, but it so much easier to use than the alternatives that it is worth the hassle.

Janus.