kramann.info
© Guido Kramann

Login: Passwort:










3.11.5.4 Modularization of VR sound
3.11.5.4 Modularization of VR sound (EN google-translate)
3.11.5.4 Modularyzacja dźwięku VR (PL google-translate)

Das vorangehende Programm wurde so einfach wie möglich umgesetzt. Jedoch sieht es dennoch kompliziert aus, weil der gesamte Quelltext in einen einzigen Tab und eine einzige Klasse geschrieben wurde.

The previous program was implemented as easily as possible. However, it still looks complicated because all the code was written in a single tab and a single class.

Poprzedni program został wdrożony tak łatwo, jak to możliwe. Jednak nadal wygląda na skomplikowaną, ponieważ cały kod został napisany na jednej karcie i jednej klasie.

Jetzt sollen die verschiedenen Codeteile sinnvoll modularisiert werden. Der Vorteil hiervon ist, dass jetzt schneller auch mehrere Klangquellen eingeführt werden können. Der Nachteil hiervon ist, dass man nicht mehr alle Programmiermöglichkeiten offen hält.

Now the different code parts are to be modularized meaningfully. The advantage of this is that now multiple sound sources can be introduced faster. The disadvantage of this is that you no longer keep all programming options open.

Teraz różne części kodu mają być znacząco zmodularyzowane. Zaletą tego jest to, że teraz wiele źródeł dźwięku można wprowadzić szybciej. Wadą tego jest to, że nie są już otwarte wszystkie opcje programowania.

Zunächst wird ein sehr ähnluches Beispiel genommen, wie im vorangehenden Programm. Jedoch soll sich nun die Schallquelle um den Betrachter herum bewegen.

First, a very similar example is taken as in the previous program. However, the sound source should now move around the viewer.

Po pierwsze, bardzo podobny przykład został wzięty jak w poprzednim programie. Jednak źródło dźwięku powinno teraz poruszać się wokół przeglądarki.

Es wird von der Möglichkeit Gebrauch gemacht, den Quelltext auf mehrere Tabs zu verteilen.

It is made use of the possibility distribute the source code to several tabs.

Wykorzystano tę możliwość rozpowszechniać kod źródłowy do kilku zakładek.

Es wird ein Tab mit dem Namen "helper" angelegt, in dem Hilfsfunktionen untergebracht werden. Dadurch wird der Quelltext an den Hauptstellen etwas übersichtlicher.

A tab called helper will be created, be accommodated in the auxiliary functions. This makes the source code at the main offices a bit clearer.

Zostanie utworzona zakładka o nazwie helper, być zakwaterowanym w funkcjach pomocniczych. Dzięki temu kod źródłowy w głównych urzędach jest nieco jaśniejszy.

import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

import android.media.AudioTrack;
import android.media.AudioFormat;
import android.media.AudioManager;

import android.media.AudioRecord;
import android.media.MediaRecorder;

//VR:
import processing.vr.*;
PShape spheres;
PMatrix3D eyeMat = new PMatrix3D(); //Object to put in the actual eye-matrix
float[] q = {0,0,-100}; //sphere position
float[] q_old = {0,0,-100}; //sphere position
float[] ear_left  = new float[3];
float[] ear_right = new float[3];
float[] left_right = new float[2];


Sound sound;
float[] mywavtone;
public void setup()
{
    mywavtone = ladeWavMix("ton69.wav");
    sound = new Sound();
    fullScreen(STEREO); //with glasses
    //fullScreen(MONO); //without glasses
    orientation(LANDSCAPE);
    
    //VR:
    spheres = createShape(GROUP);
    
    PShape sphere = createShape(SPHERE, 50);
    sphere.setStroke(false);
    sphere.setFill(color(255,0,0));
    //sphere.translate(0,0,-100);
    sphere.translate(q[0],q[1],q[2]);
    
    spheres.addChild(sphere);    
}

public void draw()
{
    //get actual position in q and put it to the sphere:
    PShape sphere = spheres.getChild(0);
    sphere.translate(-q_old[0],-q_old[1],-q_old[2]);
    sphere.translate(q[0],q[1],q[2]);
    q_old[0]=q[0];
    q_old[1]=q[1];
    q_old[2]=q[2];
  
    background(0,0,255);
    getEyeMatrix(eyeMat); //actualize eye matrix  
    translate(eyeMat.m03, eyeMat.m13, eyeMat.m23); //put view point to [0 0 0] == put world to eye-position
    lights(); //turn on lights
    shape(spheres); //render group of shapes (here only one sphere)   
}

public class Sound implements Runnable
{
    private AudioTrack audioTrack; //Connection to sound card
    private int sr=44100; //Sample rate
    private int sr_wav=44100; //Sample rate of the loaded tone. So this is handled seperately.
    //Thus, you do not care about if changing sampling rate sr. It may differ to one of sound files!
    private int buffsize=512; //buffer size
    private int buffsize2=buffsize*2;
    private ScheduledExecutorService schedExecService; //realtime process    
    private short[] shortbuffer = new short[buffsize*2]; //stereo buffer l0 r0 l1 r1 l2 r2 ... frequently sent to sound card

    private double t=0.0; //realtime
    private double dt = 1.0/(double)sr; //time step according to sampling rate.
    private double dt_wav = 1.0/(double)sr_wav; //time step according to sampling rate of loaded file.
    private float MAX_SHORT = 32767.0;
    public Sound()
    {
        try
        {                                                
            audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, sr, 
                                    AudioFormat.CHANNEL_OUT_STEREO, 
                                    AudioFormat.ENCODING_PCM_16BIT, 
                                    buffsize*10, 
                                    AudioTrack.MODE_STREAM);
            audioTrack.setStereoVolume(1.0f, 1.0f);
            
            audioTrack.play();            
        }
        catch(Exception eee)
        {
            System.out.println("FEHLER: "+eee);
        }     
        
        schedExecService = Executors.newSingleThreadScheduledExecutor();
        long period = (buffsize*1000)/sr; //Seconds per Beat==60/BPM, die Hälfte weil 8tel, mal 1000 weil Millisekunden.
        schedExecService.scheduleAtFixedRate(this, 0, period, TimeUnit.MILLISECONDS);                
        
        
    }
    public void run()
    {
        for(int i=0;i<buffsize2;i+=2)
        {
             //let the sphere move in a circle arround the listener:
             //The update takes place here, to calculate sound always in the right way.
             float x_sphere = -100.0*sin(TWO_PI*0.25*(float)t);
             float z_sphere = -100.0*cos(TWO_PI*0.25*(float)t);
          
             q[0] = x_sphere;
             q[1] = 0.0;
             q[2] = z_sphere;
          
             //ACCURACY OF TIME SHOULD BE HIGHER THAN ONE OF AMPLITUDE.
             //SO TIME IS CALCULATED USING double AND AMPLITUDE USING float.
          
             //This may be the part of the programm where you might want to modify something like
             //introducing other sound sources / directional sound and so on:
             
             //Calculate the actual sample from the provided wav-file:
             //Actual time may be in between two samples:
             float amplitude = getSampleValue(mywavtone,dt_wav,t);

             //VR begin
             
             updateEarLeftEarRight(ear_left,ear_right);             
             updateVolumeLeftRight(left_right, ear_left,ear_right,q);
             
             float volume_left  = left_right[0];
             float volume_right = left_right[1];
             
             //VR end
             
             float left  = MAX_SHORT*amplitude*volume_left;
             float right = MAX_SHORT*amplitude*volume_right;
             shortbuffer[i]=(short)left;
             shortbuffer[i+1]=(short)right;
             t+=dt; //increment of realtime for each sample
        }
      
        audioTrack.write(shortbuffer, 0,buffsize2);      
    }
}

Code 3.11.5.4-1: Androidsound004

/**
*    t==realtime
*   dt_wav == time step of one sample in wavtone
*   mywavtone == mono wavfile
*/
public float getSampleValue(float[] mywavtone, double dt_wav, double t)
{
             //Calculate the actual sample from the provided wav-file:
             //Actual time may be in between two samples:
             double T_entire = dt_wav*(double)mywavtone.length; //duration of file
             double t_file = t - T_entire*Math.floor(t/T_entire); //actual time in looped wav file
             
             int index_lo = (int)(t_file/(double)dt_wav); //first sample index
             int index_hi = (index_lo+1)%mywavtone.length; //following sample index
             
             //calculating weight for each sample:
             double t_part = t_file - dt_wav*Math.floor(t_file/dt_wav);
             double part   = t_part/dt_wav; //range [0,1]
             float weight_lo = 1.0 - (float)part;
             float weight_hi = (float)part;
             
             return  weight_lo*mywavtone[index_lo] + weight_hi*mywavtone[index_hi];  
}

public void updateVolumeLeftRight(float[] left_right, float[] ear_left,float[] ear_right,float[] q)
{
             //Normalize Position vector of sphere:
             float lq = getVectorLength(q);
             
             //Calculate different volume for left and right ear:
             float volume_left  = ear_left[0]*q[0]/lq + ear_left[1]*q[1]/lq + ear_left[2]*q[2]/lq;
             float volume_right = ear_right[0]*q[0]/lq + ear_right[1]*q[1]/lq + ear_right[2]*q[2]/lq;
             
             //cut off negative values:
             if(volume_left<0.0)
                 volume_left=0.0;
             if(volume_right<0.0)
                 volume_right=0.0;
             
             left_right[0] = volume_left;             
             left_right[1] = volume_right;             
}

public void updateEarLeftEarRight(float[] ear_left,float[] ear_right)
{
             getEyeMatrix(eyeMat); //actualize eye matrix for sounding
             //make up ez (direction of view)             
             float[] ez = {eyeMat.m02,-eyeMat.m12,eyeMat.m22};
             float[] ex = {eyeMat.m00,-eyeMat.m10,eyeMat.m20};
             //calculate direction for left ear (+PI/8), right (-PI/8) smaller than in 003:
             ear_left[0] = ez[0]-0.5*ex[0];
             ear_left[1] = ez[1]-0.5*ex[1];
             ear_left[2] = ez[2]-0.5*ex[2];
             
             ear_right[0] = ez[0]+0.5*ex[0];
             ear_right[1] = ez[1]+0.5*ex[1];
             ear_right[2] = ez[2]+0.5*ex[2];
                          
             float length_ear_left  = getVectorLength(ear_left);
             float length_ear_right = getVectorLength(ear_right);
             
             ear_left[0]/=length_ear_left;
             ear_left[1]/=length_ear_left;
             ear_left[2]/=length_ear_left;
             
             ear_right[0]/=length_ear_right;
             ear_right[1]/=length_ear_right;
             ear_right[2]/=length_ear_right;   
}

/**
*    calculates length of a vector
*/
public float getVectorLength(float[] vec)
{
    return sqrt( vec[0]*vec[0]+vec[1]*vec[1]+vec[2]*vec[2] );
    
}
    /**
     * Loads a Stereo-Wav-Datei. It is mixed to one channel and provided as float-Array.
     */
    public float[] ladeWavMix(String name) //Gleich nur Mix aus linkem und rechtem Kanal laden (aus 44100 2x16Bit extrahieren)
    {
            int zl,zh,lo,gesamt;
            float gross = 0.5f*(255.0f*256.0f + 255.0f);          
            byte[] dat = loadBytes(name);
            int anz = dat.length;          
            float[] y = new float[(anz-44)/4];                        

            int inx=44;
            
            for(int i=0;i<y.length;i++)
            {
                 zl = dat[inx++];
                 zh = dat[inx++];
                 if(zl>127)
                     zl-=256;
            
                 if(zh>127)
                     zh-=256;

                 lo     =  zl;
                 if(lo<0)
                     lo+=256;
                 gesamt = (zh+128)*256;                
                 gesamt+=lo;
                
                 y[i] = 0.5f*((float)gesamt - gross)/gross;        

                 zl = dat[inx++];
                 zh = dat[inx++];
                 if(zl>127)
                     zl-=256;            
                 if(zh>127)
                     zh-=256;

                 lo     =  zl;
                 if(lo<0)
                     lo+=256;
                 gesamt = (zh+128)*256;
                
                 gesamt+=lo;
                
                 y[i] += 0.5f*((float)gesamt - gross)/gross;                         
            }                
            return y;    
    }    

Code 3.11.5.4-2: helper-tab in Androidsound004

Androidsound004.zip -- sketch folder.