java sound on linux: how to capture from TargetDataLine quickly enough to keep up? -


i'm using java sound api , java 1.7. having difficulty reading targetdataline enough keep being recorded when run application on linux (java version "1.7.0_51", java(tm) se runtime environment (build 1.7.0_51-b13), java hotspot(tm) 64-bit server vm (build 24.51-b03, mixed mode), red hat enterprise linux 5). don't have problem when running same program on windows 7 laptop. i'm stumped.

to isolate issue, wrote program captures targetdataline interval of time (interactively determined) , records amount of time spent in blocking read of fixed number of bytes each time, prints these out along mean read time, total time elapsed, , time worth of audio captured.

my test program follows:

import java.io.bufferedreader; import java.io.ioexception; import java.io.inputstreamreader; import java.util.arraylist; import java.util.linkedlist; import java.util.list;  import javax.sound.sampled.audioformat; import javax.sound.sampled.audiosystem; import javax.sound.sampled.dataline; import javax.sound.sampled.lineunavailableexception; import javax.sound.sampled.mixer; import javax.sound.sampled.targetdataline;  /**  * test of mic capture delay given buffer , fetch settings.  */ public class miccapturedelaytest {     /**     * audio format used capturing , transmitting     */    private static final audioformat format =          new audioformat(8000, 16, 1, true, true);     /**     * target data line buffer size request, in bytes.     */    private static final int mic_buffer_size = 1000;     /**     * number of bytes try fetch target data line @     * time.     */    private static final int mic_fetch_size = 480;     /**     * searches available mixers on system have microphone.     * @return list of matching mixers     */    private static list<mixer.info> findmicrophonemixers() {       mixer.info[] mixerinfos = audiosystem.getmixerinfo();       list<mixer.info> matches = new arraylist<>();       (mixer.info mixerinfo : mixerinfos) {          mixer mixer = audiosystem.getmixer(mixerinfo);          dataline.info lineinfo = new dataline.info(targetdataline.class,                format);          boolean issupported = mixer.islinesupported(lineinfo);           if (issupported) {             matches.add(mixerinfo);          }       }        return matches;    }     /**     * test recording thread.     */    private static class micfetcher extends thread {        /**        * requested recording state.        */       private boolean shouldrecord = false;        /**        * current processed recording state of thread.        */       private boolean isrecording = false;        /**        * java audio interface line microphone data captured from.        */       private targetdataline linefrommic;        /**        * runs test mic capture thread body.        */       @override       public void run() {           list<mixer.info> matchingmixerinfo = findmicrophonemixers();           // use first matching mixer.          mixer mixertouse = audiosystem.getmixer(matchingmixerinfo.get(0));           dataline.info info = new dataline.info(targetdataline.class, format);           try {             linefrommic = (targetdataline) mixertouse.getline(info);             linefrommic.open(format, mic_buffer_size);          } catch (lineunavailableexception e) {             e.printstacktrace();             return;          }           byte[] transferbuffer = new byte[mic_fetch_size];          list<long> readtimesnanos = new linkedlist<>();          int numframescaptured = 0;          long starttimenanos = 0;           while (true) {             boolean currentshouldrecord;             synchronized(this) {                currentshouldrecord = shouldrecord;             }              if (!isrecording && currentshouldrecord) {                // start recording.                 system.out.println("starting.");                linefrommic.start();                isrecording = true;                starttimenanos = system.nanotime();              } else if (isrecording && !currentshouldrecord) {                // stop recording.                system.out.println("stopping.");                linefrommic.stop();                linefrommic.flush();                 system.out.print("read times (ms): ");                long sumreadtimesnanos = 0;                int = 0;                (long sampletimenanos : readtimesnanos) {                   if (i % 5 == 0) {                      system.out.println();                   }                   system.out.printf("%.2f  ", sampletimenanos / 1.0e6);                   sumreadtimesnanos += sampletimenanos;                   ++i;                }                system.out.println();                system.out.println(                      "mean read time (ms): "                            + (sumreadtimesnanos / 1.0e6                                  / readtimesnanos.size()));                 long stoptimenanos = system.nanotime();                system.out.println("time captured (s): "                      + (numframescaptured / format.getframerate()));                system.out.println("time elapsed (s): "                      + (stoptimenanos - starttimenanos) / 1.0e9);                 readtimesnanos.clear();                numframescaptured = 0;                isrecording = false;              } else if (isrecording) {                // continue recording.                 long beforetimenanos = system.nanotime();                 // retrieve data line.  blocks.                int numbytesread = linefrommic.read(                      transferbuffer, 0, mic_fetch_size);                numframescaptured += numbytesread / format.getframesize();                 long aftertimenanos = system.nanotime();                long timeelapsednanos = aftertimenanos - beforetimenanos;                readtimesnanos.add(timeelapsednanos);             }          }       }        /**        * requests toggle recording state of test recording thread.        */       public synchronized void togglestate() {          shouldrecord = ! shouldrecord;       }    }     /**     * runs test program.  newline toggles state.     * @param args command line args-- none needed     * @throws ioexception if thrown when trying console input     */    public static void main(string[] args) throws ioexception {       bufferedreader inputreader =             new bufferedreader(new inputstreamreader(system.in));        micfetcher fetcher = new micfetcher();       fetcher.start();        while (true) {          // toggle state each line of input (ie, press enter toggle).          inputreader.readline();          fetcher.togglestate();       }    } } 

when run in linux environment, 10-second recording, output looks like:

starting.  stopping. read times (ms):  54.00  18.10  36.62  36.32  35.99   18.10  18.25  54.26  18.30  35.56   18.12  35.51  36.74  17.22  36.70   35.29  18.33  35.60  18.23  54.72   19.00  37.99  18.14  18.37  53.91   18.37  35.34  36.00  18.00  36.00   18.00  54.71  17.22  18.12  36.18   36.64  36.08  18.00  54.34  18.26   18.27  35.44  18.30  54.77  18.33   18.24  36.51  35.47  36.52  18.35   17.14  54.96  18.13  36.73  17.21   54.95  18.28  18.37  36.54  36.72   35.56  18.37  17.23  54.46  18.36   35.53  18.08  36.00  36.00  17.99   54.30  18.06  35.22  18.00  18.00   53.93  18.32  35.63  36.64  18.16   35.21  18.30  55.65  18.23  18.35   35.55  36.32  35.60  18.30  36.33   36.21  17.22  36.54  18.32  54.96   17.19  18.36  35.62  36.67  35.25   18.29  18.37  54.63  18.37  36.54   18.35  53.91  18.37  17.23  36.70   36.09  36.01  17.19  18.33  53.91   18.37  36.56  18.36  35.53  36.58   18.16  53.84  18.26  36.03  18.08 18.12  54.24  18.08  36.14  36.19 18.12  36.08  18.11  53.80  18.28 18.37  36.55  18.13  53.99  18.00 36.12  35.54  18.28  36.56  17.20 53.96  18.00  18.01  36.67  36.53 36.71  17.19  18.37  54.37  18.02 35.97  18.00  54.00  18.00  18.00 36.00  35.99  36.34  18.37  18.35 53.93  18.13  36.63  18.33  36.33 36.34  18.33  36.55  35.51  36.66 18.29  18.06  54.00  17.99  36.08 18.25  36.64  36.38  18.37  35.55 36.66  18.21  36.73  17.19  54.27 18.13  35.55  18.18  36.31  35.56 18.34  53.90  18.36  18.09  36.15 18.22  53.90  18.32  18.37  53.89 18.19  36.04  17.20  53.94  18.31 18.37  36.55  36.70  36.61  18.35 17.18  53.97  18.32  36.55  19.01 18.99  57.00  18.99  38.01  18.98 38.00  18.99  36.99  36.35  18.37 36.55  36.70  18.04  38.00  19.00 38.00  37.99  18.99  37.99  19.00 37.06  36.43  36.03  18.00  18.00 54.47  18.25  36.70  18.22  18.37 53.55  18.33  35.59  36.59  18.29 35.36  18.37  54.89  18.24  36.44 18.33  18.36  53.52  18.13  36.36 35.57  18.20  35.52  18.20  53.78 18.18  18.16  35.49  36.67  36.54 18.37  36.53  36.67  17.19  36.65 18.29  54.87  17.14  18.24  36.68 35.49  35.61  18.27  18.36  53.77 18.24  35.43  18.35  53.90  18.37 18.24  38.00  38.00  37.99  18.99 19.01  37.98  19.00  57.00  18.99 19.00  38.00  18.99  55.01  18.98 35.99  18.00  18.01  54.98  18.00 37.00  17.99  36.00  36.00  17.99 54.01  18.98  18.00  36.02  18.98 53.16  18.34  35.59  36.20  17.98 36.00  18.00  54.00  17.99  18.00 36.00  35.99  36.01  17.99  18.00 54.00  17.98  35.99  18.00  54.28 mean read time (ms): 30.210176811594206 time captured (s): 10.35 time elapsed (s): 10.466399 

the output similar 10 second recording in windows environment looks like:

starting.  stopping. read times (ms): 44.96  30.13  29.97  29.97  30.04 29.96  29.96  30.00  29.99  30.00 29.92  30.01  30.02  30.01  29.99 29.85  45.12  30.03  29.92  29.96 29.98  30.00  29.98  30.00  0.24 44.73  29.94  30.04  29.96  29.86 29.96  30.05  29.85  30.17  30.02 30.00  29.94  29.99  29.99  30.04 29.97  44.99  29.99  30.08  29.88 30.05  29.95  29.97  29.87  0.15 44.95  29.98  29.91  30.08  29.98 30.00  30.01  29.96  29.94  30.04 30.01  29.96  29.88  30.00  29.95 30.04  44.99  29.99  29.96  30.03 30.00  30.07  29.94  30.01  0.21 44.77  29.95  30.02  30.01  30.00 29.96  29.98  30.00  30.00  29.94 29.99  30.04  29.93  29.99  30.02 29.98  44.99  29.99  29.96  30.01 30.03  29.95  30.00  29.97  0.21 44.81  29.88  30.05  29.99  29.99 30.01  29.97  29.99  29.99  29.98 29.99  30.00  29.97  29.98  29.97 30.01  44.95  29.97  30.03  30.00 30.00  30.00  29.99  29.97  0.21 44.79  29.95  30.00  29.99  29.95 29.98  29.93  30.06  29.94  30.08 29.97  30.00  29.97  29.99  29.98 29.94  45.05  30.04  29.91  30.00 29.99  29.97  30.01  29.98  0.21 44.79  29.94  29.99  29.89  30.06 30.03  29.96  30.04  29.98  29.90 30.04  30.00  29.98  30.00  29.97 30.07  44.96  29.98  29.93  30.07 29.98  29.90  30.00  29.94  0.13 44.97  29.98  29.99  29.94  30.02 30.00  29.93  29.99  30.02  30.01 29.99  29.96  30.02  29.90  29.93 30.01  45.04  30.06  29.99  29.98 29.94  30.04  30.00  29.92  0.20 44.83  29.94  29.99  30.00  30.01 30.02  29.87  30.03  29.94  30.03 29.99  30.00  30.07  29.90  29.95 30.05  44.97  30.01  29.98  29.97 30.01  29.99  30.00  29.97  0.21 44.77  29.96  30.00  30.03  29.91 30.00  30.01  30.03  29.93  29.98 29.99  29.99  29.93  30.04  30.04 30.01  44.92  30.04  29.97  29.91 30.08  29.89  29.97  29.88  0.15 45.01  30.09  29.89  30.01  30.01 29.97  29.95  29.96  30.05  30.04 29.88  30.00  29.99  29.94  30.05 29.98  44.99  30.01  30.00  29.99 29.95  30.00  29.88  30.11  0.21 44.78  30.01  29.96  29.99  29.98 29.98  29.99  30.01  29.91  29.82 30.10  29.99  30.15  29.96  29.93 29.98  45.05  29.97  29.99  30.02 29.96  29.98  29.95  30.04  0.21 44.74  30.02  29.97  29.97  30.03 29.99  29.93  29.94  30.07  29.99 29.99  29.94  30.02  29.97  29.90 30.01  45.12  29.91  30.03  29.95 30.03  29.97  29.87  30.09  0.20 44.79  29.98  29.97  29.99  30.01 30.01  29.97  29.99  29.99  30.01 29.99  29.94  30.01  30.00  29.98 29.98  45.02  29.97  29.91  30.06 29.99  29.96  30.02  29.98 mean read time (ms): 30.073811959885386 time captured (s): 10.47 time elapsed (s): 10.777957116 

summary stats on linux environment 30-second recording:

mean read time (ms): 30.152922254616133 time captured (s): 30.87 time elapsed (s): 31.135111 

summary stats on windows environment 30-second recording:

mean read time (ms): 30.020078674852652 time captured (s): 30.54 time elapsed (s): 30.901762071 

i'm noticing difference between time elapsed , time captured increases increasing recording time on linux side. looks individual fetch times less regular on linux side.

i've tried adjusting buffer , fetch sizes, haven't found combination allows quick enough fetching line.

what cause slowness in fetching? how determine reasonable fetch , buffer sizes such there low latency quick enough fetching keep real time? there possible sound configuration issues on linux affect or should check?

thanks!

private static final int mic_fetch_size = 480; // 0.12 seconds of data  

this far small buffer size reliable performance. @ 16 bit mono, represents 240 sound samples. make more 16000 samples, or:

private static final int mic_fetch_size = 32000; // 2 seconds of data 

note: java sound not guarantee amount read, , instead return number of bytes are read. point is, allow opportunity read up to 2 seconds of data (if available).

i think should solve of problems described above.


Comments

Popular posts from this blog

android - Get AccessToken using signpost OAuth without opening a browser (Two legged Oauth) -

org.mockito.exceptions.misusing.InvalidUseOfMatchersException: mockito -

google shop client API returns 400 bad request error while adding an item -