aws fun
aws transcribe
(from #gcu 28/8/2020)
Le service AWS Transcribe permet de realiser la transcription de contenu audio, c'est à dire l'extraction de texte à partir d'une source audio. Ce service utilise du deep learning.
Mise en oeuvre
Afin d'utiliser ce service en ligne de commande, il est nécessaire de disposer d'un compte AWS de type “Programmatic”, c'est à dire de disposer d'une aws_access_key et d'une aws_secret_access_key.
Il est nécessaire de disposer d'un droit d'accès AWS S3 Full Access pour uploader les vidéos sur le bucket S3 et d'un AWS Transcribe FullAccess pour lancer des taches de transcription.
Il faut disposer d'un accès à un bucket s3 qui servira d'espace de stockage pour les documents à transcrire.
Le traitement se réalise en trois étapes.
Configuration d'un profil pour l'accès à l'API via awscli
aws configure --profile s3wtfake
Téléchargement de la vidéo à transcrire vers un bucket S3
aws --profile s3wtfake s3 cp _YPTuniqid_5f4297c4aa6946.62176812_SD.mp4 s3://wtfake-transcribe/
Lancement d'une tache de transcription
aws --profile s3wtfake transcribe start-transcription-job --language-code fr-FR --transcription-job-name CDLXX4 --media MediaFileUri=s3://wtfake-transcribe/_YPTuniqid_5f4297c4aa6946.62176812_SD.mp4
Lister les taches de transcription en cours
aws --profile s3wtfake transcribe list-transcription-jobs
Le retour de la commande sera sous cette forme :
{
"TranscriptionJobSummaries": [
{
"TranscriptionJobName": "CDL49",
"CreationTime": 1598560456.056,
"StartTime": 1598560456.076,
"LanguageCode": "fr-FR",
"TranscriptionJobStatus": "IN_PROGRESS",
"OutputLocationType": "SERVICE_BUCKET"
},
{
"TranscriptionJobName": "CDLXX1",
"CreationTime": 1598560519.323,
"StartTime": 1598560519.342,
"CompletionTime": 1598561094.367,
"LanguageCode": "fr-FR",
"TranscriptionJobStatus": "COMPLETED",
"OutputLocationType": "SERVICE_BUCKET"
},
Récupération de l'état de la transcription
aws --profile s3wtfake transcribe get-transcription-job --transcription-job-name CDL49
{
"TranscriptionJob": {
"TranscriptionJobName": "CDL49",
"TranscriptionJobStatus": "COMPLETED",
"LanguageCode": "fr-FR",
"MediaSampleRateHertz": 44100,
"MediaFormat": "mp4",
"Media": {
"MediaFileUri": "s3://wtfake-transcribe/_YPTuniqid_5f29b633a4b738.59288279_SD.mp4"
},
"Transcript": {
"TranscriptFileUri": "https://s3.eu-west-3.amazonaws.com/aws-transcribe-eu-west-3-prod/087911240729/CDL49/1ec8eebf-c211-4dca-b33b-43d0186b6df6/asrOutput.json?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEAwaCWV1LXdlc3QtMyJHMEUCIFY3hEl6zWCQpGO34vgE1TdXKE0fK0aw9xvMua4KlXx8AiEAjqE5LU%2FPCx4GGKLPnpl2p%2FZK3e4SzgqH0z25EcqeelYqtwMI9f%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARACGgwxNzQwODA2NTMxOTgiDKhDEo%2FjUjukcllmEiqLA22HrD2VeB5QfG4l83WtwVE8OnqHQxT8Euns%2BQHL8PNSE0dOkH8C%2FMxs%2FsAYxQYBvKCV%2BQdKMW4%2FNFIssH86TwQSaUnVhDatEKdFO43LCUirK3wYWDUxfsFvRVyUxsiW8j8e9l9H6UuLbOWmuaP86Ixn%2BiEkTM3LEJi%2BZ%2BMalVo8U0R7LLS8w6ieZJsty4LJBzyJ0tcehC9cvX8CwQhiSi43mpY7iBQNsDYjKhV6o5bWk3fdWtEP1rBKicCBjlr8eoGkBswGtZ1Q3LtWxUCqCxmAA5%2FgAk04KL0ir4qmNfUfrrvMaUiGoi2qmDNtVmO9foDXdaWVggSeb0nmEHYlobA3w%2FAyamUM7pY4XPLFYNHBD6nx9o0dVH0lVpOvGSSjtlCANVWpcVy%2F7RMVjN5K00Ndc0ZVD6IR3vA6WIu5uCMgggZuPKyb8Wm1OWoQCmnWOmq%2FSLDnBrEHBRfm%2FTFoNqwtZAtNkVjjBzRb9QQ872q%2B8oAEy8ES%2BsRqn50Y0C6wFQOqqD4KcfHIsYLzMLuToPoFOusBs%2BQz3enUEe%2FMB%2FuyeGgf1AH%2BriScJWHPE45DEUCztTaTbxoL5DxPUFZcEbZt4mpsVLAv%2F9M1UkxVrfl%2BzczNBkMrZgPFW8rZhn3h4BcUZjn3LDIKbjnIq5yLXSbTm6qdLo3MBL7QBFPqoUzn2HGVSvCnOHx70zSU8Rct2umuklYIDmnZ3vWCcWRkaO34kKwWhgyRPAxCGszG4dZy42w%2BYcyFDaQA52T0snJRLhWNEhI24t26GdzegaJ%2BmNXUUV066igMng7WWSLkHvGyWrBGKUYDIWfDkxAQZYjquLLJzVv2y9C%2FfY7lDx3FTA%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200827T210501Z&X-Amz-SignedHeaders=host&X-Amz-Expires=900&X-Amz-Credential=ASIASRCAD66HPFKJIF5W%2F20200827%2Feu-west-3%2Fs3%2Faws4_request&X-Amz-Signature=d538a8b816e3ebad1e9aca2e60835ed3b1ce695c6a3d7e584a0abef228b58cd0"
},
"StartTime": 1598560456.076,
"CreationTime": 1598560456.056,
"CompletionTime": 1598561834.293,
"Settings": {
"ChannelIdentification": false,
"ShowAlternatives": false
}
}
}
Récupération du fichier JSON contenant la transcription :
wget 'https://s3.eu-west-3.amazonaws.com/aws-transcribe-eu-west-3-prod/087911240729/CDL49/1ec8eebf-c211-4dca-b33b-43d0186b6df6/asrOutput.json?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEAwaCWV1LXdlc3QtMyJHMEUCIFY3hEl6zWCQpGO34vgE1TdXKE0fK0aw9xvMua4KlXx8AiEAjqE5LU%2FPCx4GGKLPnpl2p%2FZK3e4SzgqH0z25EcqeelYqtwMI9f%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARACGgwxNzQwODA2NTMxOTgiDKhDEo%2FjUjukcllmEiqLA22HrD2VeB5QfG4l83WtwVE8OnqHQxT8Euns%2BQHL8PNSE0dOkH8C%2FMxs%2FsAYxQYBvKCV%2BQdKMW4%2FNFIssH86TwQSaUnVhDatEKdFO43LCUirK3wYWDUxfsFvRVyUxsiW8j8e9l9H6UuLbOWmuaP86Ixn%2BiEkTM3LEJi%2BZ%2BMalVo8U0R7LLS8w6ieZJsty4LJBzyJ0tcehC9cvX8CwQhiSi43mpY7iBQNsDYjKhV6o5bWk3fdWtEP1rBKicCBjlr8eoGkBswGtZ1Q3LtWxUCqCxmAA5%2FgAk04KL0ir4qmNfUfrrvMaUiGoi2qmDNtVmO9foDXdaWVggSeb0nmEHYlobA3w%2FAyamUM7pY4XPLFYNHBD6nx9o0dVH0lVpOvGSSjtlCANVWpcVy%2F7RMVjN5K00Ndc0ZVD6IR3vA6WIu5uCMgggZuPKyb8Wm1OWoQCmnWOmq%2FSLDnBrEHBRfm%2FTFoNqwtZAtNkVjjBzRb9QQ872q%2B8oAEy8ES%2BsRqn50Y0C6wFQOqqD4KcfHIsYLzMLuToPoFOusBs%2BQz3enUEe%2FMB%2FuyeGgf1AH%2BriScJWHPE45DEUCztTaTbxoL5DxPUFZcEbZt4mpsVLAv%2F9M1UkxVrfl%2BzczNBkMrZgPFW8rZhn3h4BcUZjn3LDIKbjnIq5yLXSbTm6qdLo3MBL7QBFPqoUzn2HGVSvCnOHx70zSU8Rct2umuklYIDmnZ3vWCcWRkaO34kKwWhgyRPAxCGszG4dZy42w%2BYcyFDaQA52T0snJRLhWNEhI24t26GdzegaJ%2BmNXUUV066igMng7WWSLkHvGyWrBGKUYDIWfDkxAQZYjquLLJzVv2y9C%2FfY7lDx3FTA%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200827T210501Z&X-Amz-SignedHeaders=host&X-Amz-Expires=900&X-Amz-Credential=ASIASRCAD66HPFKJIF5W%2F20200827%2Feu-west-3%2Fs3%2Faws4_request&X-Amz-Signature=d538a8b816e3ebad1e9aca2e60835ed3b1ce695c6a3d7e584a0abef228b58cd0'
Nettoyage
Il est nécessaire de liberer l'espace utilisé par les taches afin de ne pas se voir (sur)facturer des ressources non utilisées :
aws --profile s3wtfake transcribe delete-transcription-job --transcription-job-name CDL49 aws --profile s3wtfake s3 rm s3://wtfake-transcribe/_YPTuniqid_5f29bb94220821.62922269_SD.mp4