Capturing in-car audio allows developers to interact with users via raw audio data provided to them from the car's microphones. In order to gather the raw audio from the vehicle, we must leverage the PerformAudioPassThru
RPC.
PerformAudioPassThru does not support automatic speech cancellation detection, so if this feature is desired, it is up to the developer to implement.
Before starting audio capture, the app has to subscribe to AudioPassThru
notification. SDL provides audio data as fast as it can gather it, and sends it to the developer in chunks. In order to retrieve this audio data, observe the OnAudioPassThru
notification:
sdlManager.addOnRPCNotificationListener(FunctionID.ON_AUDIO_PASS_THRU, new OnRPCNotificationListener() { @Override public void onNotified(RPCNotification notification) { OnAudioPassThru onAudioPassThru = (OnAudioPassThru) notification; byte[] dataRcvd = onAudioPassThru.getAPTData(); processAPTData(dataRcvd); // Do something with audio data } });
This audio data is only the current audio data, so the developer must be in charge of managing previously retrieved audio data.
To initiate audio capture, we must construct a PerformAudioPassThru
object. The properties we will set in this object's constructor relate to how we wish to gather the audio data from the vehicle we are connected to.
PerformAudioPassThru performAPT = new PerformAudioPassThru(); performAPT.setAudioPassThruDisplayText1("Ask me \"What's the weather?\""); performAPT.setAudioPassThruDisplayText2("or \"What's 1 + 2?\""); performAPT.setInitialPrompt(TTSChunkFactory.createSimpleTTSChunks("Ask me What's the weather? or What's 1 plus 2?")); performAPT.setSamplingRate(SamplingRate._22KHZ); performAPT.setMaxDuration(7000); performAPT.setBitsPerSample(BitsPerSample._16_BIT); performAPT.setAudioType(AudioType.PCM); performAPT.setMuteAudio(false); sdlManager.sendRPC(performAPT);
AudioPassThru
notification listener should be added before sending PerformAudioPassThru
request or else some audio data may be missed.
In order to know the currently supported audio capture capabilities of the connected head unit, please refer to the SystemCapabilityManager
. It can retrieve the AudioPassThruCapabilities
that the head unit supports.
Currently, Ford's SYNC 3 vehicles only support a sampling rates of 16 khz and a bit rate of 16.
AudioPassThru
is a request that works in a different way when compared to other RPCs. For most RPCs a request is followed by an immediate response that informs the developer whether or not that RPC was successful. This RPC, however, will only send out the response when the Perform Audio Pass Thru is ended.
Audio Capture can be ended in 4 ways:
AudioPassThru
has timed out.
If the audio passthrough has proceeded longer than the requested timeout duration, Core will end this request and send a PerformAudioPassThruResponse
with a Result
of SUCCESS
. You should expect to handle this audio passthrough as though it was successful.
AudioPassThru
was closed due to user pressing "Cancel".
If the audio passthrough was displayed, and the user pressed the "Cancel" button, you will receive a PerformAudioPassThruResponse
with a Result
of ABORTED
. You should expect to ignore this audio pass through.
AudioPassThru
was closed due to user pressing "Done".
If the audio passthrough was displayed, and the user pressed the "Done" button, you will receive a PerformAudioPassThruResponse
with a Result
of SUCCESS
. You should expect to handle this audio passthrough as though it was successful.
AudioPassThru
was ended due to the developer ending the request.
If the audio passthrough was displayed, but you have established on your own that you no longer need to capture audio data, you can send an EndAudioPassThru
RPC.
EndAudioPassThru endAPT = new EndAudioPassThru(); sdlManager.sendRPC(endAPT);
You will receive an EndAudioPassThruResponse
and a PerformAudioPassThruResponse
with a Result
of SUCCESS
, and should expect to handle this audio passthrough as though it was successful.
To process the response that we received from an ended audio capture, we monitor the PerformAudioPassThruResponse
by adding a listener to the PerformAudioPassThru
RPC before sending it. If the response has a successful Result
, all of the audio data for the passthrough has been received and is ready for processing.
performAPT.setOnRPCResponseListener(new OnRPCResponseListener() { @Override public void onResponse(int correlationId, RPCResponse response) { Result result = response.getResultCode(); if(result.equals(Result.SUCCESS)){ // We can use the data }else{ // Cancel any usage of the data Log.e("SdlService", "Audio pass thru attempt failed."); } } });