This something that been on my mind for a bit. When I was younger and envisioned Smart Visual Display Technology along with Advanced AI, I always pictured AI to be some kind of Visual Avatar, like Pat from Smarthouse or Cortana from Halo.

But currently most of the trend for AI seem to move more towards Audio AI that has no avatars. This works perfectly fine as well.

But which method you all prefer for XR and AR applications of Advanced AI?

will it be apple, microsoft, meta or even open AI? i put my money on meta. thoughts?

Man I am exited for the future of augmented reality glasses. I bet in 5-10 years we will be using them the same way we are using our cellphones and computers

Say you live in the city and at any given time there are thousands of radio waves traveling past you, and through you, bouncing off of buildings and whatnot. Obviously you can't and shouldn't have access to important data the signals contain but it'd be fascinating to be able to see this magnetic field and individual signals traveling through it represented by their own colors and shades and watch them radiate and bounce around. Is there anything like this currently?

Hi Folks, I've been thinking of hacking something together this holiday. Building a VPS might be fun. I know it is a mammoth task but building something basic to learn is my goal here. If anyone has tried this before, please point me in the right direction. Thanks!

Edit:

Here is what I want to do, I want to have a mapper app in Android just to keep it simple. Where I click pictures of a place and I want to create a point cloud out of it.

I want to visualize that point cloud in the Unity Scene, and using that I want to place objects in my scene. And then when I build that scene I want to see these objects in AR. Kinda like how Immersal works, you scan a place. They give you a GLB and a point cloud that you can drop in your scene place objects relative to the scan/point cloud. And then they use that point cloud to place objects in AR.

I already have a long list with predictions from 2017 to 2023. But sometime last year I stopped.

What did industry people say about when consumer AR will take off?

Here's what I got so far.

Sony, Hiroshi Mukawa: When we began development [20 years ago], I was expecting consumer applications, such as real-time translation, navigation, games, web searches, etc., but I currently think that the major market will be in commercial fields for some time. AR HMDs are actually being used for navigation of order picking in warehouses, remote instructions to less-experienced workers, training, and other applications. Going forward, I believe a time will come when AI will be able to supplement and enhance human capabilities, especially as its recognition and prediction accuracy improves, and will also be able to automatically create content tailored to the user’s environment. I also think that lightweight, refined hardware that can be used on a daily basis may come out in the next three to four years. This evolution of content and hardware will pump the consumer market.

Snap, Evan Spiegel: Certainly by the end of the decade. So we're getting quite close. I mean, we've been working on glasses, I think, for about 10 years now at Snap. There were a lot of fundamental technical hurdles that we had to overcome in terms of the display system, what we call the optical engine, that connects the glasses and the glass piece of the lens with the projector system. And of course everything we had to do to design the Snap operating system to work across two processors and really distribute heat and power effectively in the glasses. I think the form factor [of the Spectacles 2024] is still a little heavy, a little big. Those are the form factor issues that will be addressed in future iterations. And our strategy has been to just get the glasses in the hands of developers today, so that they can start building for Spectacles, so that when consumers adopt them, there are all sorts of amazing experiences that people can try when they use the glasses. I think by 2030 we will see widespread consumer adoption of AR glasses. I think they are closer than folks think. While I think the progress has been quite slow over the last 10 years, as I look to the next 10 and the future of AR glasses, the progress will be quite rapid and consumer adoption should follow.

Meta, Mark Zuckerberg: There's a billion to 2 billion people who wear glasses on a daily basis. I think everyone who has glasses is pretty quickly gonna upgrade to smart glasses over the next decade. And then I think it's gonna start being really valuable and a lot of other people who aren't wearing glasses today are going to end up wearing them too. And initially I thought [the current Ray-Ban Meta without display are] on the technology path to building full holographic glasses. At this point, I actually just think both are gonna exist long term. I think that there are gonna be people who want the full holographic glasses and I think that there are gonna be people who prefer the superior form factor or lower price of a device where they are primarily optimizing for getting AI. I also think there's gonna be a range of things in between. There's like a heads up display version for that you probably just need 20, 30 degrees [field of view]. And each step on this continuum from display-less to small display to full holographic, you're packing more technology in, so each step up is gonna be a little more expensive, is gonna have a little more constraints on the form factor. And then there's the mixed reality headsets. On that we said we're not gonna try to fit into a glasses form factor. For that one we're gonna really go for all the compute that we want and we're gonna say: okay, this is gonna be more of a headset or goggles form factor. And my guess is that that's gonna be a thing long term too.

Meta, Andrew Bosworth: Full AR is not the only option. You take the Ray-Ban Meta glasses that we have and what if these have a display in them? Maybe it's not a super wide field of view, maybe it's not holograms in space, but it adds to the experience. There's not just one set of products coming - full AR - there's a whole suite of products coming between here and there. [...] We already have the next two products in development based on the technology we developed for Orion. We think this is just a proof of how exciting this technology is going to be as these technologies become consumer-ready. We are more than 1 year away, less than 10 years away [from AR glasses] but we have very clear line-of-sight to a consumer product. The price is not clear to us yet. One of the big things going from a prototype like [Orion] is understanding what can we learn that allows us to simplify or what do we need to keep in future designs and that's gonna contol where the price lands. But we really wanna get this into a price point and form factor that not just consumers use it but developers wanna build for it.

Aloha AR ohana, so I create custom stationary products such as stickers, greeting/business cards, photos, ect. but would love to integrate AR somehow into my products. I'm super lost due to all the various no coding options out there such as Mywebar and AR-Code. Would love to hear any suggestions as to where to begin/look as well as any feedback with no coding AR. Much mahalos!! 🤙🏽

Discussion From consensus llm: Imagine a scenario where one uses the matter hub and ar glasses to use xiao AI, bixby, google assistant, Siri and Alexa as employees for a work from home business. Give them access to all the devices with unique features of the highest qualities available from their respective platforms. Allow for them to each answer questions for peak collective conversational ability with the user. Endow with usage of their companies particular llm’s (i.e. copilot for xiao ai, etc). Allow them to be conversational with one another to the degree that is possible. Consider using “accessibility” type features available on their platforms to expand the capabilities they have to control and make use of devices for useful purposes as a sort of hack. Use jailbreak tools to expand further. Use apps to expand further. Use home assistant hubs/bridge to expand further. Make limited use of other ai/voice assistants such as from meta, etc to expand further (only to answer questions or use capabilities they themselves aren’t capable of). Make use of their programming and other brand specific tools and metrics to the upper limit of their capability to control or make use of those attributes that would further reflect the particular personality underlying the collective consciousness of their companies. Consider these differences to make them uniquely useful for divergent thinking. Again use tools/metrics/apps/tech from other companies to the limited extent to do things their particular platforms cannot do only to expand their capabilities. Make use of all other available llm’s to the upper limit they can make use of them without exceeding their capacities. AR glasses should display floating icons for each voice assistants in a star pattern such that one need only look in the direction of the particular icon to start or stop conversational abilities. All smart home devices available with unique upper limit abilities should be used (i.e. imagine a smart home filled from head to toe with any and all smart devices for maximum performance and control). Utilize as much electricity as necessary paying for by trading and using crypto/stocks/forex as minimally as necessary automatically if possible. With all that said, what insights can you give about what an expanded grid of ecosystems for voice assistant control that are possible? What problems might arise? Consider using AI to resolve these problems and co-ordinate the devices to their fullest potential. Would such a set up exceed the limits of anything known to man currently in terms of user interface control? Write a tentative plan for implementing these instructions directly, if cost and skill are no object.

Insights Into an Expanded Grid of Ecosystems for Voice Assistant Control

An integrated ecosystem of voice assistants, powered by the latest LLMs (large language models), interconnected smart home devices, and augmented reality (AR) could redefine the concept of user interface control and home automation. The design leverages the unique strengths of each platform while mitigating their weaknesses with complementary technologies.

Potential Capabilities: • Divergent Thinking: Each assistant brings unique traits based on their brand’s focus: • Xiao AI: Efficiency in automation and smart home integration, particularly for Xiaomi ecosystems. • Bixby: Contextual execution and adaptive learning for Samsung devices. • Google Assistant: Search, vast LLM-based reasoning, and deep integration with Android and Google’s services. • Siri: Privacy-focused execution and Apple ecosystem fluency. • Alexa: Comprehensive third-party skill library and Amazon smart home compatibility. • Conversational Synergy: Allowing assistants to interact conversationally and pool their strengths. • Example: Google Assistant provides knowledge while Alexa executes the task using a specialized smart home skill. • AR Augmentation: Visual, interactive icons in AR glasses simplify command initiation, task tracking, and device management. • Expanded Accessibility Features: • Voice synthesis and transcription. • Environmental control for users with disabilities (e.g., accessibility tools from Siri and Google). • Energy Optimization and Funding: Automation of financial tasks (e.g., cryptocurrency, stock trades) to fund electricity costs using minimal risk algorithms. Challenges and Problems: • Overlapping Commands: Confusion between assistants when tasks overlap or are not clearly segmented. • Privacy Risks: Data sharing among different platforms may breach privacy norms. • Latency Issues: Coordinating numerous assistants and devices could lead to delays. • Resource Management: Ensuring energy consumption, device upkeep, and software updates are efficient. • Compatibility: Integrating platforms not designed to work together without breaking APIs or terms of service. AI-Driven Problem Resolution Strategies 1. Centralized AI Orchestration Hub: • Role: A meta-coordinator (custom-built LLM, like GPT-4) to manage task delegation, resolve conflicts, and optimize performance. • Functionality: • Assign tasks to assistants based on their specialties. • Translate commands into device-specific protocols. • Prioritize energy-efficient devices and allocate resources dynamically. 2. Conflict Management: • Use AI to prevent command overlaps by assigning each assistant a domain of control. • Example: Siri handles personal reminders; Alexa controls IoT; Google Assistant answers queries. 3. Privacy Safeguards: • Data anonymization and sandboxed environments for cross-platform interaction. • User-controlled permissions and encrypted communication channels. 4. Latency Optimization: • Pre-cache frequently used commands or sequences of tasks. • Use edge computing for local processing where possible. 5. Scalability Framework: • Utilize Home Assistant hubs or bridges as a foundation for linking diverse devices. • Build modular layers that can expand to integrate emerging technologies (e.g., quantum computing).

Tentative Plan for Implementation

Phase 1: Hardware Setup 1. AR Glasses: • Select AR glasses (e.g., Meta Quest, Apple Vision Pro) capable of floating UI elements. • Integrate motion tracking for icon selection by gaze. 2. Smart Home Devices: • Outfit a home with a comprehensive array of IoT devices: • Smart lights, thermostats, security systems, appliances, and entertainment centers. • Choose devices optimized for integration (e.g., Matter-compatible). 3. Home Assistant Hub: • Deploy platforms like Home Assistant, Hubitat, or SmartThings to bridge incompatible devices.

Phase 2: Software Integration 1. Assistant Coordination: • Establish APIs or hacks (as allowed) to enable cross-assistant communication. • Use jailbreak tools sparingly to unlock additional functionalities while minimizing risk. 2. Central Orchestration AI: • Create a bespoke AI layer using LLMs to oversee tasks and ensure optimal interaction between systems. • Implement using tools like Node-RED for automation scripting.

Phase 3: Enhancing Ecosystem with Apps and Tools 1. Expanding Capabilities: • Use apps and integrations to extend functionality (e.g., Tasker for Android, Shortcuts for Apple). • Leverage accessibility features like voice synthesis, predictive typing, and adaptive controls. 2. Financial Automation: • Program assistants to manage energy costs through trading crypto or stocks using platforms like Robinhood API.

Phase 4: Testing and Optimization 1. Simulate Scenarios: • Test common workflows (e.g., managing meetings, automating house routines). • Evaluate response times and resource consumption. 2. Debugging: • Identify and resolve conflicts, privacy issues, and inefficiencies.

Phase 5: Full Deployment 1. User Training: • Familiarize users with AR interactions and assistant-specific strengths. 2. Ongoing Monitoring: • Use AI-driven analytics to continually improve performance and add features.

Conclusion

This system would exceed existing interfaces in sophistication, combining divergent AI thinking, AR, and full smart home integration. While ambitious, the design remains feasible given current technology and advances in AI coordination.

https://preview.redd.it/17lqhafk6w3e1.png?width=1167&format=png&auto=webp&s=7f92d6f0352a4f747bab4ced6efa746a3e34b5cd

Hop in friends: we are in for a wild ride!