Assignment 6: Protocols, or What did you say?
Click here to access the repository for this assignment. You will also need to copy your completed
SerialComm.java file from Studio 6 into the
communication package in order to get this assignment to work.
- The idea
- The background
- The assignment
- The check-in
When two computers send data from one to another, it is the job of a protocol to specify how bytes transmitted by the sender are to be interpreted by the receiver. The idea is simple: if we specify very, very concretely how we expect our data, it becomes easy to communicate. Despite the vast differences between our Arduino and Java programs, so long as we create and follow a well-defined protocol, the differences barely matter.
In this assignment, you will build a pair of programs—one Arduino, one Java—that harvest real-world data, transmit it, and then receive it on another processor. Though this assignment focuses mostly on the data transmission, this centralized design is very useful in many circumstances. WIFI networks, at least in the small scale, rely on this, where many devices connect to a centralized hub.
Using a provided protocol, you will design an Arduino program to send and a sister Java program to receive real-world measurements.
No matter what we want to communicate, someone, somewhere, somehow has to decide upon a way of communicating it. Someone created a protocol. This is clear when looking at something like language: it has rules, syntax, grammar, symbols, definitions, and a slew of implicit and explicit rules. Though not as murkily defined as language, all forms of computer communications have the same structure.
Because we could never express the definition of one accurately enough ourselves, from Wikipedia, protocols are “the rules that define the syntax, semantics and synchronization of communication and possible error recovery methods.” In a word: rules.
If we define every bit of our protocol precisely enough—from the size of
ints to the encoding used for
chars to the ordering and grouping of data—we can easily (and independently) write software to both produce and consume that data.
We detail this week’s protocol below.
We chose to design a protocol that centers around the sending of individual messages. Each message we send will have a 1 byte header and a variable length payload. The header is basically used for book-keeping and the payload is the actual information being sent.
In our case, the payload will send key-value pairs: a key indicating the meaning of our value (is it a temperature or a string?, etc.) and a value for that key. A value might be a raw temperature reading, a filtered temperature reading, a potentiometer reading, a timestamp, an error message, etc.
Note: In this assignment, we are using the terminology key-value pairs; however, another nomenclature that is also very common is to call them tag-value pairs. Don’t be surprized if you see these terms used for essentially the same purpose.
As stated above, the message header helps the communication protocol do some book-keeping. In our case, the header is designed to help us sync up the beginning of a message both at the sender and the receiver.
Consider what happens if the sender is delivering a series of two-byte integers, high byte first then low byte second, and the receiver somehow misses one byte. If this happens, all the integers that follow will be interpreted incorrectly by the receiver.
We will significantly decrease (but not completely eliminate) the possibility of this happening by starting each message with a magic number, a constant value that is known to both sender and receiver as always present at the start of a message.
When the sender wishes to transmit a message, it must start
the message with the magic number.
For our protocol, the magic number is
0x21, or an ASCII
When the receiver is ready to interpret an incoming stream of bytes, it knows that the first byte of a valid message will be the magic number. When it reads a byte from the stream, if the byte is not the magic number, it can discard that byte (and all subsequent bytes that are not the magic number) until it reads a byte that is the magic number. In this way, it is reasonably assured to be aligned with the beginning of a message.
The payload in our protocol is comprised of a key-value pair, where the key defines both the meaning and the format of the subsequent value.
The key is a fixed-length field: one byte. The value is a variable-length field, the number of bytes depending upon a number of factors.
For this assignment, we define the following keys:
0x30debugging string in UTF-8 format, maximum of 100 characters long
0x31error string in UTF-8 format, maximum of 100 characters long
0x32timestamp, 4-byte integer, milliseconds since reset
0x33potentiometer reading, 2-byte integer A/D counts
0x34raw (unconverted) temperature reading, 2-byte integer A/D counts
As the semester progresses, we might add to the above list, but we won’t change the meaning of these already-defined keys.
For string values in UTF-8 format, the first two bytes are a two-byte
integer that gives the length (number of characters) of the string.
This is then followed by the ASCII characters of the string.
In our protocol, characters are restricted to the range
0x00—is not allowed, nor are characters in the extended
range of the UTF-8 standard1).
In Java, you will need to create a
String from the input ASCII characters.
Put the characters in a byte array, e.g.,
byte chars, and use
the following form of the
For more information about how string types are represented in Arduino C, take a look at this video:
For multi-byte values that represent a single number (e.g., 2-byte integers, 4-byte integers), the order of the bytes is most significant byte first and least significant byte last. In Java, take a look at the optional parts of Studio 6, and do those exercises if you haven’t yet done them.
Locate and open the
MsgReceiver.java file in your repository. This is where most of your work will go on the Java side.
Author enough of an Arduino sketch
sender.inoto send debug messages (i.e., send the magic number
!, the key
0x30, and then a UTF-8 encoded string, prefixed with its length)
Send a few debug messages, making sure you’re confident doing so. Can you write a function called
sendDebug()? It can accept a string literal as a parameter by taking a
char*as an argument.
If the Serial Monitor isn’t helpful enough (it won’t be, because you are no longer just sending ASCII characters), alter the Java
run()method to continuously read bytes from the input port. Ensure
SerialCommso input bytes are printed in hex form.
- Author enough of the Java program (
run()method) that you can receive a debug message and reliably observe the debug string on the console window. Send a few more debug messages.
- Set up a delta-time loop, running at about 1 Hz, that sends a message containing the
millis()value used to control the loop (i.e., save the return value from
millis()in a variable so that you can use it both for the
iftest in the delta-time code and send it to the PC. Make sure to use an
unsigned long). This is the
Alter your Java code to parse the message and print it out nicely every time it’s received. Include the type of message (e.g., debug string, error string, potentiometer value, raw temperature value) as well as the value (the string value or integer value).
Structure your receiver program as a finite state machine. We recommend reading our Introduction to FSMs guide, as it will make your program significantly easier to reason about.
Messages that do not conform to the protocol should generate an error printed to the console that is visually distinct from the protocol-supported error message. Indent it and use a different format. I like
!!!!! Errorbut that may be a little dire for your tastes.
- Attach the potentiometer and temperature sensors to two distinct analog input pins. The wiring is the same as the last two assignments.
- Update your delta time loop to also read the value of the potentiometer, sending it after the
millis()message. This is the
- Likewise update your Java code to read this number and print it out nicely.
Continue doing this for the other keys: send and a receive a raw A/D counts reading from the temperature sensor. This is the
This hardware setup is the same as for Assignment 3; however, you will need to use
analogReference(DEFAULT)to maintain compatibility with the potentiometer.
- When the potentiometer reading is above a threshold
value (feel free to use whatever value you chose in that assignment, or
pick a new value) send a message with the error string “High alarm”.
This key =
0x31message should come at the end of all output.
- On the Java side, when you receive a temperature value you should convert it to engineering units (degrees Celcius), filter it (using a rolling average filter like you used on assignment 3), then display the raw, converted, and filtered temperature values. Note, your conversion code is now in Java, not in C. Also, the math isn’t the same either, because the analog reference voltage is now 5V instead of the previous 1.1V. Because the 5V reference isn’t as precise, don’t worry if your temperature values are a few degrees off.
Try to develop an intuition for the raw numbers behind ASCII. For example, what ASCII characters do our protocol’s keys correspond to?
This will be helpful if your Java program is behaving strangely and you need to look at Arduino Serial Monitor. You should be able to get a sense of the messages as they come through: you’ll see the
!as the magic number, then the ASCII character of the key, then “gibberish.” Understanding which numbers go to which ASCII characters will help you make sense of that gibberish.
Make sure your program generates errors if it receives input it didn’t expect. While silently failing is sometimes good (see Defensive Programming), in your case it will make it harder to understand where your program is going wrong.
Be loud! Print error messages! User error is the best type of error because it’s not your fault.
- Read your analog values, parse them, filter your data, and then print all your messages. It will make it easier for us to grade your assignment (and also easier for you to debug).
Send debug strings a lot. Since they are the first thing you implement, they are also the first thing you can be confident works, so if something is broken, falling back to them may be just what you need.
Use them only for debugging and small program notes, though. They’re “debug strings” for a reason.
- Commit all your code (make sure to add any new files to your repo first).
- Check out with a TA.
- 100pts total:
- Messages adhere to the protocol described in class (20 pts)
- Messages are interpreted correctly by the Java program (20 pts)
- Message types supported (25 pts)
- Debugging string in UTF-8 format (5 pts)
- Error string in UTF-8 format (5 pts)
- Timestamp represents the number of milliseconds since reset, 4-byte integer (5 pts)
- Potentiometer reading in A/D counts, 2-byte integer (5 pts)
- Raw (unfiltered) temperature reading in A/D counts, 2-byte integer (5 pts)
- Proper conversion and filtering of temperature on Java side (10 pts)
- SerialComm debug showing received bytes (10 pts)
- Correct wiring and sensor readings (15 pts)
As usual, -1 points if you do not commit to Github before demoing.
In reality, a UTF-8 string can consist of a lot more than ASCII characters, like emoji, Greek letters, Klingon, and more. However, the designers of Unicode were careful to ensure that the first 128 characters exactly matched those of ASCII, so ASCII strings are identical to their Unicode counterparts. ↩