Amidst the severe labor shortage in global manufacturing and construction industries, making heavy equipment like excavators autonomous has become a focal point for the industry. Startup company Bedrock Robotics recently successfully addressed one of the most time-consuming challenges in developing autonomous systems—massive video data annotation—by collaborating with the AWS Generative AI Innovation Center.

image.png

Traditionally, training construction automation systems requires manual annotation of millions of hours of surveillance videos to identify various buckets, hooks, and work tasks, which is not only extremely costly but also highly inefficient. AIbase reported that by introducing visual language models (VLMs) from the Amazon Bedrock platform, the team was able to let AI automatically recognize and describe complex scenes in videos through natural language instructions, just like chatting with a human.

This "AI annotating AI" model brought a significant leap in efficiency. The report noted that due to unusual shooting angles and dust interference on construction sites, the recognition accuracy of construction tools for ordinary models was only 34%, but after targeted prompt engineering optimization, the accuracy soared to 70%. This means that the previously cumbersome manual screening process has now been transformed into an automated and scalable data pipeline.

Currently, this technology has been applied in the Bedrock Operator