How to handle the PTS correctly using Android AudioRecord and MediaCodec as audio encoder? -


i'm using audiorecord record audio stream during camera capturing process on android device. since want process frame data , handle audio/video samples, not use mediarecorder.

i run audiorecord in thread calling of read() gather raw audio data. once data stream, feed them mediacodec configured aac audio encoder.

here of codes audio recorder / encoder:

m_encode_audio_mime = "audio/mp4a-latm"; m_audio_sample_rate = 44100; m_audio_channels = audioformat.channel_in_mono; m_audio_channel_count = (m_audio_channels == audioformat.channel_in_mono ? 1 : 2);  int audio_bit_rate = 64000; int audio_data_format = audioformat.encoding_pcm_16bit;  m_audio_buffer_size = audiorecord.getminbuffersize(m_audio_sample_rate, m_audio_channels, audio_data_format) * 2; m_audio_recorder = new audiorecord(mediarecorder.audiosource.mic, m_audio_sample_rate,                                    m_audio_channels, audio_data_format, m_audio_buffer_size);  m_audio_encoder = mediacodec.createencoderbytype(m_encode_audio_mime); mediaformat audio_format = new mediaformat(); audio_format.setstring(mediaformat.key_mime, m_encode_audio_mime); audio_format.setinteger(mediaformat.key_bit_rate, audio_bit_rate); audio_format.setinteger(mediaformat.key_channel_count, m_audio_channel_count); audio_format.setinteger(mediaformat.key_sample_rate, m_audio_sample_rate); audio_format.setinteger(mediaformat.key_aac_profile, mediacodecinfo.codecprofilelevel.aacobjectlc); audio_format.setinteger(mediaformat.key_max_input_size, m_audio_buffer_size); m_audio_encoder.configure(audio_format, null, null, mediacodec.configure_flag_encode);

i found the first time of audiorecord.read() takes longer time return, while successive read() have time intervals more close real time of audio data. example, audio format 44100hz 16bit 1channel, , buffer size of audiorecord 16384, full buffer means 185.76 ms. when record system time each call of read() , subtracting them base time, following sequence:

time before each read(): 0ms, 345ms, 543ms, 692ms, 891ms, 1093ms, 1244ms, ...

i feed these raw data audio encoder above time values pts, , encoder outputs encoded audio samples following pts:

encoder output pts: 0ms, 185ms, 371ms, 557ms, 743ms, 928ms, ...

it looks encoder treats each part of data having same time period. believe encoder works correctly since give raw data same size (16384) every time. however, if use encoder output pts input of muxer, i'll video audio content being faster video content.

i want ask that:

  1. is expected first time of audiorecord.read() blocks longer? i'm sure function call takes more 300ms while records 16384 bytes 186ms. issue depends on device / android version?
  2. what should achieve audio/video synchronization? have workaround measure delay time of first call of read(), shift pts of audio samples delay. there better way handle this?

convert mono input stereo. pulling hair out time before realised aac encoder exposed mediacoder works stereo input.


Comments

Post a Comment

Popular posts from this blog

android - Get AccessToken using signpost OAuth without opening a browser (Two legged Oauth) -

i using signpost oauth access data magento server. have read various tutorials on same , reach point open browser user can enter credentials. however, per requirement have automate part. hence, user should not browser page. have set-up done on server side ( magento ), hit url , returned call page. same through program in android. i have tried below, consumer = new commonshttpoauthconsumer(key, secret); provider = new commonshttpoauthprovider(oauth_init_url,access_token_url, authorize_url); try { provider.retrieverequesttoken(consumer, oauth.out_of_band); log.d("tokens" , consumer.gettoken() + " -- " + consumer.gettokensecret()); } and request tokens. dont know how bypass next step. tried directly accessing accesstoken (stupid of me) provider.retrieveaccesstoken(consumer, "mycallback://callback"); no luck, ends in oauth.signpost.exception.oauthnotauthorizedexception: authorization failed (server replied 401). ...
Read more

org.mockito.exceptions.misusing.InvalidUseOfMatchersException: mockito -

getting error org.mockito.exceptions.misusing.invaliduseofmatchersexception: invalid use of argument matchers! 1 matchers expected, 2 recorded. exception may occur if matchers combined raw values: //incorrect: somemethod(anyobject(), "raw string"); when using matchers, arguments have provided matchers. example: //correct: somemethod(anyobject(), eq("string matcher")); at when( getprogramservice .callservice(any(getprogramtype.class))) .thenreturn(jaxbresponse); please explain error , possible resolution the code posted looks fine; don't see problem it. code before or above causing exception. key here: invalid use of argument matchers! 1 matchers expected, 2 recorded. any , used above, doesn't return "special kind of null" or "special kind of getprogramtype"; don't exist. instead, returns null , side effect puts matcher on stack. whe...
Read more

google shop client API returns 400 bad request error while adding an item -

i'm working on solution add items in google shopping market java application. following tutorial real mess because of missing classes of laste release, after setting 1.5-beta maven managed make base example typecheck correctly classes. still can make simple task of adding 1 item. instead of posting whole code i'm posting part creates product feed. private product createproduct(string id) { product product = new product(); product.title = "red wool sweater"; product.content = new content( "text", "comfortable , soft, sweater keep warm on " + "cold winter nights. red , blue stripes."); product.appcontrol = new appcontrol(); product.appcontrol.addrequireddestination("productads"); product.externalid = id; product.lang = "it"; product.country = "it"; product.condition = "nuovo"; product.price = new price("eu...
Read more