23

What I am trying to do: use Android's MediaCodec to encode raw PCM audio samples into a raw AAC file.

The problem I have: when I use FFMPEG to pack the generated raw AAC file into an M4A container, FFMPEG complains about missing codec parameters in the file.

Details:

Since I can't find any MediaCodec sample code for the audio encoder that generates an output AAC file, I tried to modify the video encoder into an audio encoder. The original code is here: source_code

I configured the audio encoder like this:

 mEncoderFormat = MediaFormat.createAudioFormat("audio/mp4a-latm", (int)mAudioSampleRate, 2); // redundant? mEncoderFormat.setString(MediaFormat.KEY_MIME, "audio/mp4a-latm"); mEncoderFormat.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectELD); mEncoderFormat.setInteger(MediaFormat.KEY_SAMPLE_RATE, kSampleRates); mEncoderFormat.setInteger(MediaFormat.KEY_BIT_RATE, kBitRates); mEncoderFormat.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 2); testEncoderWithFormat("audio/mp4a-latm", mEncoderFormat); try { codec.configure( mEncoderFormat, null /* surface */, null /* crypto */, MediaCodec.CONFIGURE_FLAG_ENCODE); } catch (IllegalStateException e) { Log.e(TAG, "codec '" + componentName + "' failed configuration."); return; } Log.d(TAG, " testEncoder configured with format = " + format);

Then I feed the encoder with 10ms worth of PCM samples per frame. The encoder takes each frame, generates a frame of bitstream, and I write the bitstream into an FileOutputStream. The loop continues until the end of the input file.

The code runs to the finish. I do 'adb pull' to get the generated AAC file from the device to my PC, and use FFMPEG to read it. Below is the command and the error FFMPEG spits out:

$ ffmpeg -f aac -i BlessedNoColor_nexus7_api18.aac ffmpeg version N-45739-g04bf2e7 Copyright (c) 2000-2012 the FFmpeg developers built on Oct 20 2012 00:20:36 with gcc 4.7.2 (GCC) configuration: --enable-gpl --enable-version3 --disable-pthreads --enable-runt ime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libass - -enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enab le-libgsm --enable-libmp3lame --enable-libnut --enable-libopenjpeg --enable-libo pus --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheo ra --enable-libutvideo --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-li bvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --ena ble-zlib libavutil 51. 76.100 / 51. 76.100 libavcodec 54. 67.100 / 54. 67.100 libavformat 54. 33.100 / 54. 33.100 libavdevice 54. 3.100 / 54. 3.100 libavfilter 3. 19.103 / 3. 19.103 libswscale 2. 1.101 / 2. 1.101 libswresample 0. 16.100 / 0. 16.100 libpostproc 52. 1.100 / 52. 1.100 [aac @ 00000000002efae0] channel element 2.0 is not allocated [aac @ 00000000003cf520] decoding for stream 0 failed [aac @ 00000000003cf520] Could not find codec parameters for stream 0 (Audio: aac, 0 channels, s16): unspecified sample rate Consider increasing the value for the 'analyzeduration' and 'probesize' options [aac @ 00000000003cf520] Estimating duration from bitrate, this may be inaccurate BlessedNoColor_nexus7_api18.aac: could not find codec parameters

My questions:

  1. I've configured the encoder before calling codec.start(). Why does the generated AAC file lack the codec parameters?
  2. In the original video codec example, parameters "csd-0" are passed from the encoder to decoder, but are not written into the bitstream file explicitly. Do I need to write them into the AAC file explicitly?
  3. I divide the input PCM samples into 10ms per frame, which does not necessarily produce a complete output packet. For each input frame, I just write whatever the encoder outputs into the file. Is that a cause for concern?

Any helps will be deeply appreciated. It'd be great if there is a sample project that does what I'm trying to do here. If my source code can help you help me, I'll post it. I need to do some cleanup. Thanks!

Edit: Changed the title from "Elementary AAC file generated by MediaCodec missing codec parameters" to "How to generate the AAC ADTS elementary stream with Android MediaCodec"

Improve this question
4
  • FWIW, the CTS audio encoder test is android.googlesource.com/platform/cts/+/jb-mr2-release/tests/… Commented Sep 18, 2013 at 5:35
  • @fadden: I looked at that test too. It does not generate an output AAC file. I did try to add the code to write the bitstream into a file. The generated file has the same problem: ffmpeg does not recognize it as an AAC file. I just edited my question to add that I can't find a sample code that generates a bitstream file (AAC in my case). Commented Sep 18, 2013 at 17:30
  • After reading this post, and this post, I checked the encoder output when italic bold ` (info.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0`. The output is 4 bytes: 0xF8, 0xE8, 0x40, 00. They are not ESDS (Elementary Stream Descriptor). What are they? Commented Sep 18, 2013 at 19:05
  • How do you feed it data to encode? Commented Feb 3, 2016 at 22:49

1 Answer 1

Reset to default
49
+50

I finally generated AAC files that are playable on both the Android device and the Windows host computer. I am posting my solution here, hoping it could help others.

First, my previous assumption that the Android MediaCodec encoder generates the elementary AAC stream was not accurate. The MediaCodec encoder generates the raw AAC stream. That's why the files could not be played. The raw AAC stream needs to be converted into a playable format, such as the ADTS stream. I have changed the title of this post to reflect my new understanding. There was another post that asked a similar question, and had an excellent answer. However, a novice may not necessarily understand the brief descriptions there. I didn't quite get it the 1st time I read that post.

So, in order to generate an AAC bitstream that can be played by a media player, I started from the EncoderTest example given by fadden in his 1st comment, but modified the original code to add the ADTS header per output frame (access unit), and to write the resulting stream into a file (replaced lines 248 through 267 of the original code with the following code snippet):

if (index >= 0) { int outBitsSize = info.size; int outPacketSize = outBitsSize + 7; // 7 is ADTS size ByteBuffer outBuf = codecOutputBuffers[index]; outBuf.position(info.offset); outBuf.limit(info.offset + outBitsSize); try { byte[] data = new byte[outPacketSize]; //space for ADTS header included addADTStoPacket(data, outPacketSize); outBuf.get(data, 7, outBitsSize); outBuf.position(info.offset); mFileStream.write(data, 0, outPacketSize); //open FileOutputStream beforehand } catch (IOException e) { Log.e(TAG, "failed writing bitstream data to file"); e.printStackTrace(); } numBytesDequeued += info.size; outBuf.clear(); codec.releaseOutputBuffer(index, false /* render */); Log.d(TAG, " dequeued " + outBitsSize + " bytes of output data."); Log.d(TAG, " wrote " + outPacketSize + " bytes into output file."); } else if (index == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) { } else if (index == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) { codecOutputBuffers = codec.getOutputBuffers(); }

Outside the loop, I defined the function addADTStoPacket like this:

/** * Add ADTS header at the beginning of each and every AAC packet. * This is needed as MediaCodec encoder generates a packet of raw * AAC data. * * Note the packetLen must count in the ADTS header itself. **/ private void addADTStoPacket(byte[] packet, int packetLen) { int profile = 2; //AAC LC //39=MediaCodecInfo.CodecProfileLevel.AACObjectELD; int freqIdx = 4; //44.1KHz int chanCfg = 2; //CPE // fill in ADTS data packet[0] = (byte)0xFF; packet[1] = (byte)0xF9; packet[2] = (byte)(((profile-1)<<6) + (freqIdx<<2) +(chanCfg>>2)); packet[3] = (byte)(((chanCfg&3)<<6) + (packetLen>>11)); packet[4] = (byte)((packetLen&0x7FF) >> 3); packet[5] = (byte)(((packetLen&7)<<5) + 0x1F); packet[6] = (byte)0xFC; }

I also added code to control how to stop generating the AAC ADTS stream, but that's application specific, so I won't detail here. With all these changes, the generated AAC files can be played on the Android device, on my Windows PC, and ffmpeg is happy with them.

Improve this answer
Sign up to request clarification or add additional context in comments.

12 Comments

Man, those 7 lines saved me days of trying to figuring it out. Awesome!
This is great, @hubeir did you manage to convert the aac to m4a using ffmpeg?
There is one thing I discovered that I would like to add. We must avoid writing the very first packet deque'd to the file. This packet contains codec information that will mess up the playback in Android music players. Also don't write end of stream packet for the same reason. All that should be in the file is data packets with the ADTS headers.
Instead of just blindly skipping the first packet, check info.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG instead. (And to flexibly support other configurations, one could parse the data in the codec config buffer and use that set the frequency index and channel config in the ADTS header.)
@mstorsjo How do you parse the BufferInfo for frequencyIndex and channelConfig?
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.