A key element in the development of AR experiences is entity identification, with an effective AR system able to identify different objects in the world around you, in order to provide additional, functional insight, overlaid on screen.

We’ve seen this in the development of AR face filters and tools in social apps – the better the system is at understanding eye placement, movement, etc., the better the effects look on screen. Over time, such systems have got better and better at responding to different elements – and now Meta has released a new model and dataset which could help to take things to the next stage.

As you can see in the above clip, Meta’s new ‘Segment Anything’ project aims to provide researchers and developers with more means to help identify objects in frame.

As per Meta:

“We are releasing both our general Segment Anything Model (SAM) and our Segment Anything 1-Billion mask dataset (SA-1B), the largest ever segmentation dataset, to enable a broad set of applications and foster further research into foundation models for computer vision.”

The process, ideally, will provide more capacity to power AR experiences, as noted, though Meta also says that it will have application in AI and VR creation processes as well.

“We anticipate that composable system design, enabled by techniques such as prompt engineering, will enable a wider variety of applications than systems trained specifically for a fixed set of tasks, and that SAM can become a powerful component in domains such as AR/VR, content creation, scientific domains, and more general AI systems.”

There’s a range of ways in which the dataset could be used, and it could be a big step in assisting in the broader development of AR models – while for Meta specifically, it could help to build out its Project Aria smart glasses project, which may or may not have been shelved due to staff cuts at the app.

Last June, The Information reported that Meta had delayed the planned release of its AR glasses as part of broader cost-cutting measures at the company. According to the report, Meta opted to scrap the first iteration of its AR wearables, which were set to hit the market next year, in favor of focusing on the second generation of its AR device, which now have no release date in frame.

Meta, of course, launched its ‘Ray Ban Stories’ smart glasses in 2021, which seemed like a precursor to its next push into AR wearables – which, at the time, also seemed like they might be coming soon. But tougher economic conditions, and its huge investment in the metaverse, seem to have derailed the plan, which have now seen its wearable ambitions put on the backburner, at least to some degree.

We don’t know how much that project has been set back, but this new dataset does seem to point to its ongoing development on this front, which could see the release of another version of its smart glasses sometime in future.

We just don’t know when, and with VR remaining its big investment focus, and generative AI now coming into frame, it has seemed like AR has been the big loser in resource allocation, at several of the big tech platforms.

But maybe, it’s coming. Maybe, developments like this point to that next stage, where full, interactive, engaging AR experiences will soon be a reality – or maybe it’s all set to be tied into VR, and merging your real and online experiences into a more immersive process.

Either way, it’s an interesting development, and the dataset release could help the broader dev community in building next-gen experiences.

You can read more about Meta’s Segment Anything project here.