IBC2023: Adobe adds AI-powered enhanced speech to Premiere Pro

Adobe has announced a host of updates to its products, including an AI-powered speech enhancement tool to Premiere Pro.

This is among a number of updates the company has announced for IBC, including improvements to text-based editing and colour options. It has also revealed additions to After Effects, including a 3D workplace, and an AI-powered rotoscoping tool. This comes after the company announced its plans for AI in its video tools at NAB Show earlier this year.

Enhance Speech allows Premiere Pro users to clean up speech affected by issues such as a poorly placed mic, excessive background noise, low-quality archive footage, and more. It uses AI to remove background noise and improve quality, with a slider so that users can decide how much of the background noise they want to keep.

In addition, Audio Category Tagging uses AI to determine which clips contain dialogue, which contain music, and which are sound effects or ambient noise. When each audio clip is selected, the most relevant tools are automatically displayed. 

Text-based editing now includes filler word detection to automatically identify “ums” and “uhs” in dialogue. Users can remove those filler words, pauses or any other unwanted dialogue in one step using bulk delete.

Premiere Pro also has a number of performance optimisations, new colour preferences and improved tone mapping, new consolidated settings in the Lumetri Color panel as well as improved LUT management and relinking, and can ensure colours look as you expect in QuickTime Player by adjusting the new Viewer Gamma option.

There is also new project templates so you can get started faster, custom export locations, a new metadata and timecode burn-in effect, batch selection of markers, and more.

Meanwhile, After Effects has also undergone a number of changes. Chief among these is a true 3D workspace, which allows users to add depth and dimension to compositions in a unified space with native import of 3D models. Image-Based Lighting places models into a scene with complementary lighting and shadows using any image as a light source, aiming to make objects appear as though they naturally belong there.

3D motion graphics are generated using an advanced 3D renderer, and 2D and 3D can be combined in a compostition with effects that reference another layer, such as Displacement Map, Vector Blur, or Calculations, able to use a 3D model layer as a source to create highly stylized renders.

Moreover, an AI model has been added to the roto brush tool, which helps to make sure difficult objects such as overlapping limbs, hair, and transparencies are separated from their background.

Finally, Frame.io has introduced Frame.io Storage Connect, which enables direct connection to AWS S3 storage with lightweight proxies available in Frame.io. it will be available to Enterprise customers later this year.

In addition, Frame.io’s comparison viewer now supports viewing video, audio, photo, design file, and PDF assets side-by-side. 

All of the features across Adobe’s programmes are currently in beta, and will be more widely available later this autumn.