Engineering Projects/Music projects/Howard Community College/fall2011/502 MusicTeam

Problem Statement

The idea is to create an instrument, similar to the on in this video. The can has already been mounted by previous teams, and there are references to code that had the hall effect probe recognizing when a magnet passed by it.

Our portion of the project involved playing a note on the keyboard with the arduino and midi shield portion of the code.

PROBLEM STATEMENT:BUILD AND CREATE A PORTABLE SOUNDS FROM ARDUINO.

Team Members

Sunny Patel

Dominick Moreno

User:Tophat

Summary

Over our time with this project, we essentially came up with a way to make sure the arduino is actually talking to your computer to properly upload code. We tried taking apart a "singing" gift card in an attempt to get it to produce sound, but broke the component necessary to actually get anything to play. Additionally, we worked with MIDI protocol and its interactions with the arduino and a keyboard. We managed to play Mary Had a Little Lamb with an arduino program we wrote ourselves. We also managed to write a code that ought to have played a note based on the voltage read on an analogpin (however, it did not actually play these notes). Lastly, we worked on mounting the tin can to the plastic board through which we will mount the Hall Effect Probes.

Team Weekly Reports:

Poster

Story

Week 1 was spent learning basic arduino interactions with software. The programming environment is similar (essentially identical) to Processing, the environment used in the previous Poppit project. Because of this, dealing with syntax wasn't much of an issue, just learning all of the various functions and their interaction with the arduino ("if", "noteOn", etc.). Difficulties with getting the arduino to work on home PCs impeded any real testing during this time. Most efforts were concentrated on studying components of the arduino environment (details are given in later summaries).

In week 2, we looked at two things. The first was getting the arduino to work on our home computers, the second was tearing apart a "singing" birthday card we bought to see if we could somehow connect it to the arduino to produce sound.

As far as the arduino is concerned, we noticed we had a problem last week when it would not upload code correctly from our home computers, but when we came to class the first day of the week, it uploaded just fine. After a fairly long process of trying several things, including checking to make sure I had the correct arduino drivers, figuring out if the programming environment was "looking" for the right board and in the right COM port, and that I had at least the minimum number of libraries to run the "Hello World" code, we finally loaded the code blinking light code into the arduino. As it turns out, it was a combination of two things:

The arduino environment (what you write code for the arduino in) has to in essence "find" the right board once it's hooked up to your computer. You have to make sure you have the right model of arduino selected (such as arduino Uno, arduino Mini, arduino Fio - in my case, arduino Duemilanove), and that it's "looking" in the correct COM port. The default, I was lead to believe, is that it will always look for the arduino Uno in COM 1. To find this setting and change it, go to tools -> board and select the correct arduino board. The correct COM port is a bit trickier if you don't already know how to find it. First one must find that the arduino is properly recognized in the device manager. On a Windows 7 64 bit operating system, go to the control panel -> system and security -> system -> device manager -> Open the drop-down menu labeled "Ports (COM & LPT)". The arduino Duemilanove was recognized as the "USB Serial Port(COM3)", but other boards are recognized under different names. To change the COM port the arduino environment looks for, go to tools -> Serial Port and select the corresponding COM point.

As far as dissecting the singing card goes, we felt rather successful in taking it apart. It consists of a small dremel board (perhaps 1 x 1 in) that seems to act as a motherboard (supplying power and "holding" code), connected to an round audio device by two wires. This audio device (now referred to as part (b)) is approximately 1 inch in diameter, and a hard green plastic dome top with a flat, thin plastic bottom. We ripped out the underside to find only a copper ring and two EXTREMELY thin copper wires running from the copper ring to the edge of part (b), where they met up with the two external wires that connect the dremel board and part (b). Additionally, we ripped out the circular looking item on the dremel board, and it was in fact a battery - as evidenced by the "3v lithium battery" written on it.

At this point, we figured we had completely destroyed the device. In the first place, it might have been possible to strip the wires connecting part (b) and the dremel board, and somehow connect that so that it can "talk" to the arduino, but when we tore open the plastic underside to part (b), we tore those incredibly thin copper wires, so it's probably broken beyond repair, or just isn't worth the effort trying to connect them back together.

For the Tuesday of Week 3, we tested the sample MIDI code just to see if the basic connection of placing the MIDI shield on the arduino would be enough to provide sound. It was, and the code used to display this can be found below. Additionally, we glued the plexiglass the tin can will be mounted on to the blocks of wood that will support it.

void setup() {
  //  Set MIDI baud rate:
  Serial.begin(31250);
}

void loop() {
  // play notes from F#-0 (0x1E) to F#-5 (0x5A):
  for (int note = 0x1E; note < 0x5A; note ++) {
    //Note on channel 1 (0x90), some note value (note), middle velocity (0x45):
    noteOn(0x90, note, 0x45);
    delay(100);
    //Note on channel 1 (0x90), some note value (note), silent velocity (0x00):
    noteOn(0x90, note, 0x00);
    delay(100);
  }
}

//  plays a MIDI note.  Doesn't check to see that
//  cmd is greater than 127, or that data values are  less than 127:
void noteOn(int cmd, int pitch, int velocity) {
  Serial.write(cmd);
  Serial.write(pitch);
  Serial.write(velocity);
}

Uploading this code to the arduino, connecting the MIDI shield to said arduino, connecting MIDI out to the MIDI shield, and MIDI in to the keyboard (or any other device with a MIDI in) will result in all notes the device (keyboard) can play (within limits), in ascending sequence. Understanding how this code works requires a basic understanding of arduino environment programming, as well as basic MIDI protocol properties.

All code using MIDI protocol MUST be set to a baud rate of 31250. This is done in the void setup with:

Serial.Begin(31250);

Additionally, this code is written to play a different note after every iteration of the loop, inside of which there is a 100 millisecond delay; in other words, every time the code repeats itself (which occurs every 1 tenth of a second), it plays a different note. To do this, a "for" statement is used. A "base" value is applied to a specified integer, and every time the loop repeats itself another value is added on to that base value, up to a certain point, at which it repeats itself. A simplified way of explaining this is to say that the for function makes the the note one value higher each time the loop repeats itself.

The code "accesses" the MIDI protocol with:

void noteOn(int cmd, int pitch, int velocity) {
  Serial.write(cmd);
  Serial.write(pitch);
  Serial.write(velocity);
}

Keyboard. Interface lights up when notes are played

This basically creates the noteOn function that is used in the void loop. It specifies what each of the three values for noteOn do - the MIDI protocol requires at least these three values to play a note, although there are others that can be used to further augment a note. Additionally, the MIDI protocol is written in hexidecimal, which is fairly easy to calculate values for using a hexidecimal calculator. "0x" always precedes any hexidecimal values, so that the code knows to read that value as hexidecimal and not decimal.

All MIDI protocol messages MUST start with the command that essentially says "these following values are to be interpreted as MIDI". This value is ALWAYS 0x90.
Pitch simply refers to what note is played. All notes have a corresponding hexidecimal value that ranges from 0 to 127/ 0 to 7F (binary/hexidecimal), where 60/3C is middle C. Finally, the third value is velocity - or how long the note is played. This is much harder to find appropriate values for, because, through testing, I found that a velocity value of 120/78 corresponds to approximately 2.88 seconds of play time. According to the previously linked page on MIDI specifications, the actual device interprets how to use velocity values. Guess and check seems to work surprisingly well, as long as the velocity value doesn't interfere with the time delay on a loop - but more on that later.

When this code was first tested, we did not understand all of the things we just examined, but rather all of this information came as a result of several hours of studying MIDI specification, and testing what code would and would not compile in the arduino environment. There were actually no problems with uploading this code, as it worked the very first time I tried connecting it all together.

The following Thursday in class we experimented with creating sound from code based on what we learned. To start, we simply wanted to play middle C over and over on a loop. The following code was written with the information extracted from the last code:

void setup() {
  Serial.begin(31250);
}

void loop() {
  noteOn(0x90, 0x3C, 0x100);
  delay(100);
}

void noteOn(int cmd, int pitch, int velocity) {
  Serial.write(cmd);
  Serial.write(pitch);
  Serial.write(velocity);
}

Where the values for velocity and delay are arbitrary. The code worked wonderfully. The next step was significantly harder, as we set out to write Mary Had a Little Lamb. The resulting (and successful) code is as follows:

void setup() {
  Serial.begin(31250);
  //MIDI protocol baud rate is 31250 bits per second
}

void loop() {
  //Plays Mary Had a Little Lamb on a loop at 145 BPM

  //Bar 1
  noteOn(0x90, 0x40, 0x28); //Quarter note: E
  delay(414);

  noteOn(0x90, 0x3E, 0x28); //Quarter note: D
  delay(414);

  noteOn(0x90, 0x3C, 0x28); //Quarter note: C
  delay(414);

  noteOn(0x90, 0x3E, 0x28); //Quarter note: D
  delay(414);

  //Bar 2
  noteOn(0x90, 0x40, 0x28); //Quarter note: E
  delay(414);

  noteOn(0x90, 0x40, 0x28); //Quarter note: E
  delay(414);

  noteOn(0x90, 0x40, 0x28); //Half note: E
  delay(828);

  //Bar 3
  noteOn(0x90, 0x3E, 0x28); //Quarter note: D
  delay(414);

  noteOn(0x90, 0x3E, 0x28); //Quarter note: D
  delay(414);

  noteOn(0x90, 0x3E, 0x28); //Half note: D
  delay(828);

  //Bar 4
  noteOn(0x90, 0x40, 0x28); //Quarter note: E
  delay(414);

  noteOn(0x90, 0x43, 0x28); //Quarter note: G
  delay(414);

  noteOn(0x90, 0x43, 0x28); //Half note: G
  delay(828);

  //Bar5
  noteOn(0x90, 0x40, 0x28); //Quarter note: E
  delay(414);

  noteOn(0x90, 0x3E, 0x28); //Quarter note: D
  delay(414);

  noteOn(0x90, 0x3C, 0x28); //Quarter note: C
  delay(414);

  noteOn(0x90, 0x3E, 0x28); //Quarter note: D
  delay(414);

  //Bar 6
  noteOn(0x90, 0x40, 0x28); //Quarter note: E
  delay(414);

  noteOn(0x90, 0x40, 0x28); //Quarter note: E
  delay(414);

  noteOn(0x90, 0x40, 0x28); //Quarter note: E
  delay(414);

  noteOn(0x90, 0x40, 0x28); //Quarter note: E
  delay(414);

  //Bar 7
  noteOn(0x90, 0x3E, 0x28); //Quarter note: D
  delay(414);

  noteOn(0x90, 0x3E, 0x28); //Quarter note: D
  delay(414);

  noteOn(0x90, 0x40, 0x28); //Quarter note: E
  delay(414);

  noteOn(0x90, 0x3E, 0x28); //Quarter note: D
  delay(414);

  //Bar 8
  noteOn(0x90, 0x3C, 0x28); //Whole note: C
  delay(1656);
  noteOn(0x90, 0x3C, 0x00); //"Silent" note that makes sure the note stops playing.
  delay(2000);

  //End song
  //Wait 2 seconds after the last note is struck to repeat
}

void noteOn(int cmd, int pitch, int velocity) { //Sets what each of the 3 values in the "noteOn" function does
  Serial.write(cmd); //All MIDI protocol messages start with 0x90 to signify the start of a MIDI message
  Serial.write(pitch); //The note. Middle C (0x3C), for example
  Serial.write(velocity); //Length of time the note is played. Values range from 0 to 127 (0x64), for example
}

We encountered several problems writing this code.

Figuring out delay values. We wanted to write this song at a certain BPM, not just get whatever BPM resulted from arbitrary delay values. We decided we wanted to see how long a single quarter note at 145 BPM played, in milliseconds. Through dimensional analysis: (145 Beats)/(1 minute) * (1 minute)/(60 seconds) * (1 second)/(1000 milliseconds) = 2.4167x10^(-3) BPMs (beats per milisecond). To calculate the length of a single beat: (1 beat) * (1 millisecond/2.4167x10^(-3) beat) = 413.793 milliseconds. Rounding up, we found that we should use a delay of 414 for quarter notes. A half note is exactly twice as long as a quarter note, and is therefore 828 milliseconds.
The biggest problem we had was actually realizing that the pitch value was a single value 0/0-127/7F (binary/hexadecimal). It was our initial belief that 0x3C was to be read as 0x[3rd octave][note C], instead of 60 (decimal). As a result we tried writing Mary Had a Little Lamb with the wrong intervals. It resulted in a "haunting" sound, that I thought to be dissonant. We incorrectly believed this was because our (initially arbitrary) velocity values were causing the notes to play longer than the delay that followed them, causing notes to overlap, and sound dissonant. This was incorrect because of A), the information just given about hexadecimal, and B), the three notes used in Mary Had a Little Lamb are all part of a common chord, and as a result would be/are not dissonant when played over one another (in fact, more "advanced" versions of Mary Had a Little Lamb for piano have the player playing those exact three notes in a different octave together with the left hand).
Another problem was dealing with velocity values. Initially, we were using a velocity value of 0x99 (hexadecimal)for the final (whole) note, with a delay of 5000 milliseconds before it would restart the loop. This resulted in the last note just not being played. I'm not sure why, but when I made the velocity value significantly (and arbitrarily) shorter (0x28), it was fine. Our theory was that the velocity value was so much longer than the decimal value, that the loop would have restarted itself before the note was done playing, and so just did not play that note. This was actually fixed by putting in the "silent" note after the final whole note.

For our final week (week 4), We were advised to work on getting a note to play when the switch on the arduino board was pressed. However, that switch turned out to be a reset button, and not necessarily a programmable switch. There is a method for programming a button, but it requires a breadboard, resistor, and separate button to work. So another method of playing a note on command.

Because the Hall Effect Probe runs with analog, we figured researching analog interactions with arduino would be useful. We found the analogRead() function.

int sensorValue = 0 //0 acts as a place holder for the variable sensorValue

void setup() {
  Serial.begin(9600); //Most analog devices require a baud rate of 9600 bits per second to operate
)

void loop() {
  sensorValue = analogRead(0); //Assigns the variable sensorValue the voltage (0v to 5v) value read (interpreted from 0 to 1023 from analog pin 0
}

As a footnote, the arduino page on analogRead said that if nothing is connected to that pin, the analogRead can pick up very random values, based on a massive amount of conditions - the one that we noticed was the closeness of one's hand. I loaded the Analog Input test code that will blink a light for a length of time based on the analogRead (the lower the voltage, the faster it blinks, the higher the voltage, the slower).

We loaded this code and found that the voltage read from pin 0 was fluctuating greatly just sitting there with seemingly no interaction with anything - however, the closeness of a hand did not seem to effect the rate at which the light blinked. We then got the idea that, because the Human brain is basically a massive ball of electrical signals, that might somehow be picked up by the arduino. I held the pin against one of our temples, and actually saw the light begin to blink slower, by my estimates around the 700-1000 millisecond range. We found this to be fairly reliable, as we tested it about fifteen times - it's a strangely reliable way to alter the analogRead input without a potentiometer.

The second part necessary to code a note playing based on conditional statements (preferably the analogRead just used) is understanding the "if" function and comparison operators. Especially when compared to the "for" function previously used in an example code, "if" is fairly simple to understand.

int sensorValue = 0;

void setup() {
  pinMode(13, OUTPUT);
  Serial.begin(9600);
}

void loop() {
  sensorValue = analogRead(0);
  if (sensorValue >= 512) { //If analogRead reads a value greater than or equal to 512
    digitalWrite(13, HIGH); //Light is on
  }
  if (sensorValue < 512) { //If analogRead reads a value less than 512
    digitalWrite(13, LOW); //Light is off
  }
}

This is a quick example we wrote using the if statement. If the analogRead picks up a voltage approximately 2.5 or greater, the LED pin is on, if it's less, the light is off. The "Serial.begin(9600);" in the voide setup is not actually necessary if the only thing being used is the analogRead (because it's not interacting with an actual analog device); so this code could actually be used in conjunction with MIDI protocol as is. So our next step was to replace the blinking light with a note playing on the keyboard. Our end result was as follows.

int sensorValue = 0;

void setup() {
  pinMode(13, OUTPUT);
  Serial.begin(31250);
}

void loop() {
  sensorValue = analogRead(0);
  if (sensorValue >= 512) { //If analogRead reads a value greater than or equal to 512
    digitalWrite(13, HIGH); //Light is on
    noteOn(0x90, 0x3C, 0x100); //Play middle C for an arbitrary length of time
    delay(sensorValue); //Pause for a value equal to what the analogRead picked up
    noteOn(0x90, 0x3C, 0x00); //"Silent" note - make sure the note stops playing
  }
  if (sensorValue < 512) { //If analogRead reads a value less than 512
    digitalWrite(13, LOW); //Light is off
    noteOn(0x90, 0x46, 0x100); //Play one octave above middle C for an arbitrary length of time
    delay(sensorValue); //Pause for a value equal to what the analogRead picked up
    noteOn(0x90, 0x46, 0x00); //"Silent" note - make sure the note stops playing
  }
  delay(5000); //Pause for 5000 seconds to ensure all notes have stopped playing
}

void noteOn(int cmd, int pitch, int velocity) { //Write the noteOn function
  Serial.write(cmd);
  Serial.write(pitch);
  Serial.write(velocity);
}

It must be noted that this code did not work as intended. It resulted in a "thumping" being played twice, a pause of approximately 4 to 5 seconds, and then repeat. Originally I thought a lack (initially) of a "silent" note to "turn off" the note being played was what was causing it, but that didn't seem to be the case. We have a list of potential things causing this:

The "silent" note being within the "if" statements could be the problem. Just taking it out of there and putting it after both if statements could work - similar to factoring in algebra. For example, 2x + 2y = 2(x+y).
Using a delay value within the if statements could somehow be causing problems. Because the sensorValue value is so difficult to control with this setup, it is essentially RNG when it is within the delay function, meaning it waits for an arbitrary length of time.
It's possible that more precise control of the program is necessary with "If... Else" statement(s) instead of just if. Further research into "If... Else" is necessary.
While a value from analogRead of 0 corresponds to 0V and 1023 to 5v, there is a level of uncertainty associated with dealing with how they interact on the interval in between. If 2.5v is not equal to 511.5, and there is some other kind of ratio involved, it is currently unknown, and could be problematic.

Material List

Figuring out what to purchase, what to build, what everything costs is a huge part of engineering. Typically there is a list of materials in stock, materials that are ordered, materials that should be ordered next time there is money, materials that have not been fully justified. These issues are part of healthy management of the engineering lab but are associated with a particular college. The detail needs to be published in this "Done" form. However in the project root, just a list of materials is necessary.

Parts necessary to finish this project:

Arduino 2009 board - used to write code
MIDI shield - used to give code access to the MIDI protocol
Keyboard with a MIDI IN attachment - used to play notes
Tin can - device used to play notes over time
Several dozen magnets - objects used as "notes" in final product
4-7 Hall Effect Probes - used to interpret the magnets as notes
Means to mount the tin can - used to hold the tin can in place
Legos to operate a hand crank to move the tin can once mounted - rotates the tin can at will to play music

Parts on hand:

Arduino 2009 board
MIDI shield
Keyboard with a MIDI IN attachment
Tin can and appropriate mounting/operating devices - Tin can is currently mounted
4-7 Hall Effect Probes

Parts ordered:

New keyboard with MIDI IN attachment - one for students to take home, another to keep in class

Possible parts necessary:

Arduino Mega board - allows for the usage of 4 different baud rates. MIDI devices such as the keyboard require a rate of 31,250 bytes per second, while analog devices such as the Hall Effect Probe require a rate of 9600.

Software List

Used software:

Arduino programming environment

Absolutely essential to programming with an arduino. A link to their site can be found here. It's open source, so free to download.

Time

We would have said we would be able to accomplish what we did in about 8-10 hours before working on the project. However, actual time spent was somewhere close to 20. We didn't actually expect learning all of the coding to take as much time as it did - the goals seemed fairly straight forward.

Tutorials

The story and team pages actually have a fairly good list of explaining everything, however, they are listed here for convenience.

The MIDI protocol
Hexadecimal Calculator
Arduino references
For those who need a stronger background in musical theory, this is not a horrible place to start.
Blinking Light Code, used throughout this project to test to see if the arduino board was properly talking to the computer.
"If" function, used to create conditional statements
"If... Else" function, used for greater control over conditional statements than just "if"
"Serial.begin" function, explains baud rate used for when the arduino interacts with various other devices
"int" function, explains the usage of integers
"analogRead" function, elaborates on how to read the voltage input from a device

Next Steps

The next team must get a note to play based on the analogRead function. This should be done more accurately with a potentiometer at first, rather than just holding the arduino to your temple. A good place to start is with breaking down the code I provided, and figuring out why it does not play a note at any given time (which it should - the if statements just have the pitch of the note different from one another, rather than the presence of a note), and then reconstruct it so that it does play a note.

From there, using previous team's knowledge, it should be possible to connect the Hall Effect Probe to the arduino, pass a magnet through its sensor, and through MIDI shield to arduino to keyboard connections, play a note. At that point, it is fairly easy to get six Hall Effect Probes working on one arduino (using the 7 analog pins provided on the arduino board), and then setting up the the actual tin can to play the notes on a hand crank.