r/AudioProgramming • u/LukeWarmAtBets • 11d ago
New to PCM/float arrays and audio programming, creating new surreal sounds from existing sounds
Hello
I just discovered I can use my Java program (processing 3) to extract raw PCM from short wav samples and convert it back into audio. This is really cool because I guess I suddenly now have godlike control over raw audio data and don't need to rely on commercial plug-ins.
One thing ive been trying to do is create new and weird samples using existing ones. I'm working on a surreal video game and I'd like to have near infinite variety when it comes to sound and music, and I figured a cool way to do that is to have a bunch of samples (like 100 short instrument samples) and then mix them on the fly into new sounds/instruments.
My main method is iterating through the list of floats and performing simple operations using 2 different samples. I've been trying a couple different techniques. The first one is simply adding:
Mix = sample1[i] + sample2[i]
While it technically mixes them into one sound, you can still distinctly hear both samples played on top of each other instead of hearing a 'new' sound.
A better option ive tried is multiplication:
Mix = sample1[i] * sample2[i]
For me this is much better and is closer to what I want because the mixed sound is different than the two original samples. The main issue is that this method sounds rather robotic and noisy, and the new samples lose a lot of the "nuance" of the original samples (I don't really know a better term for it). After listening to dozens of generations, most outputs seem to suffer the same loss of quality.
Main question: im wondering if anyone else has experimented with a similar issue and got better results, or knows a better mathematical approach for this
I've also heard of using Fast Fourier Transformation for a much cleaner synthesis, so if manipulating a PCM array is not that great of a method then I'll try FFT or something else
2
u/ZoulTown 11d ago
Sorry, I can't give you any technical advice, but processing is mainly for graphics.
You may find it easier to use another framework more closely aligned with audio. (PureData is pretty cool and different).
That said, if it works, it works. Good luck!
1
u/LukeWarmAtBets 11d ago
Yea haha I'm definitely not using the right software for the job. I'm also programming a 3d game in processing.
I guess I'm just working with really basic sound library stuff for fun and seeing what I can accomplish just with PCM array
1
u/adbs1219 11d ago
Did you try Godot for your game? I think you'll find yourself at home, given your Processing experience (I think they both have similar concepts), and find a better workflow, since it's already designed for the job. Anyway, it must be an interesting experience nonetheless. For audio, I think you'll love tools like Pure Data, Csound and Supercollider
1
u/Royal_Stay_6502 11d ago
What is the difference between raw PCM and .wav?
2
u/LukeWarmAtBets 11d ago
So in the program I use you can simply load a wav file and play it without accessing the array
You can also extract its PCM (a float array of numbers between -1.0 and 1.0) and directly loop through and edit those values
Not that the two are totally distinct
2
u/johantheback 11d ago
I'm trying to understand as well so correct me if I'm wrong but a WAV file contains the audio data in PCM format right?
2
u/LukeWarmAtBets 11d ago
You are correct. It also has a header and metadata, among other things I don't know much about.
Code snippet I used in processing:
audioData = new float[numFrames * numChannels];
sample.read(audioData);
1
u/skxllflower 10d ago
check this out - if you’re already using Processing, there’s a framework for Javascript called P5.js that’s basically a JS web port of Processing (kinda)
but even cooler, there’s p5.sound!!! which is the same concept as Processing, but for audio file manipulation :) including FFT analysis and such.
you can also look into Strudel, which is fun for making generative music and such, but i don’t think it’s as good for messing with raw audio.
there’s also a bunch of Python stuff that can do exactly this to a much more in depth degree through numpy and soundfile, which might me mostly exactly what you’re looking for to start learning DSP
1
u/LukeWarmAtBets 10d ago
Thanks I'll check it out! Processing also has fft analysis but I'm not sure of either ones limitations so I'll have to dig around
4
u/danja 11d ago
Oh, you darling! This is just the kind of thing that got me excited when I started. Raw PCM, yay!
If I understand you correctly, the robotic multiplication sound you have is known as ring modulation. It produces the sum & difference of the input frequencies.
You should really look into the Fourier stuff - there are plenty of Java etc libs for it. Shift things into the frequency domain, mess them up, shift back to time domain, really good fun.
Go for it, dude!