New Technology / Robotics
Technology signals, innovation themes, and applied engineering trends. Topic: Robotics. Updated briefs and structured summaries from curated sources.
Inside Amazon’s Potential $50B OpenAI Investment, Nvidia’s Impressive Earnings & Stock Fall
Full timeline
0.0–300.0
Amazon is proposing an investment of up to $50 billion in OpenAI, with the first $15 billion as a trial. The remaining $35 billion is contingent on OpenAI achieving artificial general intelligence or going public.
- Amazon is proposing a significant investment of up to $50 billion in OpenAI, with specific conditions attached to the funding
- The initial trial investment will be $15 billion. The remaining $35 billion is contingent on OpenAI achieving artificial general intelligence or going public
- This strategy allows Amazon to potentially benefit more from its relationship with OpenAI. This is especially important given Microsofts existing exclusivity rights with the company
- Microsoft has invested around $13 billion in OpenAI. It holds rights to resell its models and has a 20% revenue share
- The definition of artificial general intelligence remains somewhat ambiguous. Previous reports suggest that OpenAI would need to generate $100 billion in profits to meet this criterion
- An expert panel will verify when OpenAI claims to have achieved artificial general intelligence. The members of this panel have not been disclosed
- The evolving nature of artificial general intelligence definitions raises questions about the conditions set by Amazon. It also raises concerns about how these conditions align with OpenAIs future goals
300.0–600.0
Amazon and OpenAI have entered a cloud computing agreement valued at approximately $38 billion over seven years, with expectations for significant expansion. Nvidia reported a 73% revenue growth, but its stock declined due to concerns about capital expenditures and customer concentration.
- Amazon and OpenAI have agreed to a cloud computing deal worth approximately $38 billion over seven years. There are expectations for significant expansion beyond this initial agreement
- The relationship between Amazon and OpenAI includes terms that allow Amazon to benefit more if OpenAI achieves artificial general intelligence or goes public
- Nvidias recent quarterly results showed a 73% revenue growth, a dramatic acceleration compared to the previous quarter. However, the companys stock fell despite this positive news
- Concerns about Nvidias stock decline include uncertainties regarding capital expenditures at hyperscalers. There are also potential resets in component costs, which have been ongoing issues
- Nvidias gross margins are currently in the mid-70s percentage range. This is significantly higher than competitors like AMD, raising questions about the sustainability of these margins
- Customer concentration remains a reality for Nvidia, with 50% of revenues coming from large hyperscalers. This reflects the nature of IT spending in the industry
- The investment structure for Nvidia includes three installments of $10 billion. This is similar to the investment terms from a major investor, totaling $30 billion
600.0–900.0
Nvidia's H200s are in demand, particularly among smaller customers due to their compatibility with older data centers. The U.S.
- Custom ASICs, like those offered by a major tech company, are designed for specific workloads. However, the rapid evolution of AI raises concerns about selecting the right workloads
- Nvidias favorable supply contracts may end next year. This could impact their margins as component prices are expected to rise
- The depreciation schedule for chips is a significant topic of discussion. Customers generally replace old chips as new ones become available
- Nvidias H200s remain in demand, especially among smaller customers. Their compatibility with older data centers contributes to this continued interest
- The U.S. administration has approved a limited number of exports to China. However, H200s have not yet been allowed into the country
- AMDs recent investment in a provider of converged infrastructure highlights ongoing consolidation in the tech industry. This move reflects broader trends in technology partnerships
900.0–1200.0
AMD is partnering with Nutanix to enhance its presence in the enterprise market. Salesforce and Snowflake reported revenue growth of 12% and 30%, respectively, but both faced stock declines due to AI-related concerns.
- AMD is expanding its presence in the enterprise market through a partnership with Nutanix, which provides hyper-converged infrastructure solutions. This collaboration aims to enhance AMDs traction among enterprise customers
- Salesforce reported a 12% revenue growth, aided by its acquisition of Informatica. Snowflake experienced a 30% revenue growth, but both companies faced stock declines due to investor concerns about AIs impact on the software sector
- Anita Ramaswamy noted a divergence in the business models of Salesforce and Snowflake. Snowflake operates at the infrastructure layer, allowing it to assist other companies in managing data for AI workloads
- Salesforces AI product, Agent Force, currently represents about 1.7% of its projected revenue for fiscal 2027. Although it is growing rapidly, its small contribution raises questions about its overall impact on Salesforces growth
- Snowflakes free cash flow margin increased from 43% to 61% over the past year, indicating strong cash generation despite investments in AI. The company also signed its largest deal ever, valued at around $400 million
- Both companies are significant players in the software market, but the importance of their AI products remains uncertain. The acquisition of Informatica has been a key driver for Salesforces growth
1200.0–1500.0
Snowflake's revenue growth is projected to decelerate to around 27% for fiscal 2027, indicating a slowdown compared to previous quarters. Major tech companies are increasingly issuing debt to fund AI investments, raising concerns about their credit profiles.
- Snowflakes revenue growth is projected to decelerate to around 27% for fiscal 2027. This is still strong but slower than previous quarters
- Anita Ramaswamy noted that both Snowflake and Salesforce are facing challenges due to AIs impact on the software sector. This is affecting investor sentiment
- Salesforces growth rate is around 12%, which is its fastest in several years. However, its AI product, Agent Force, remains a small part of overall revenue
- Ramaswamy discussed how major tech companies like Alphabet, Amazon, and Meta are increasingly issuing debt to fund their AI investments. This raises questions about their credit profiles
- Credit analysts indicated that a downgrade for these companies is unlikely. They are projected to maintain a strong debt-to-EBITDA ratio well below the threshold
- Meta has the lowest credit rating among these companies. This is primarily due to its reliance on advertising revenue, which presents more risk compared to its competitors
- Demand for debt from these tech giants is influenced by their credit ratings. However, market dynamics also play a significant role in investors willingness to purchase their debt
1500.0–1800.0
Demand for hyper-scaler debt remains high, with recent debt offerings being oversubscribed. Saronic is raising up to $1.5 billion at a $7.5 billion valuation, focusing on autonomous warships.
- Demand for hyper-scaler debt remains high, with recent debt offerings being oversubscribed. However, there are concerns that flooding the market with too much debt could change this dynamic
- Investors may perceive the underlying credit health of companies differently than credit rating agencies do. This divergence could lead to an increase in the cost of capital for these firms
- Saronic is raising up to $1.5 billion at a $7.5 billion valuation, focusing on autonomous warships. The company aims to sell these naval vessels primarily to the U.S. Navy
- Saronics revenue profile shows it generated just over $200 million last year. Investors expect significant growth in the coming years, or the company may appear overvalued
- A venture capital firm is leading the funding round for Saronic, marking its first major investment in defense technology. This move suggests a potential trend of venture capital firms expanding into this sector
- The technical expertise required for investing in defense technology is substantial. Investors need to understand complex technologies and market dynamics to make informed decisions
1800.0–2100.0
Cliner Perkins is leading a funding round for Saronic, a startup focused on autonomous warships, marking a shift towards defense technology investments. Investors are cautious about AI startups lacking sustainable business models, preferring established companies like SpaceX.
- Cliner Perkins is leading a significant funding round for Saronic, a startup focused on building autonomous warships. This marks a notable move for the firm, which has not heavily invested in defense technology before
- Investors are becoming increasingly cautious about funding AI startups, particularly those lacking a sustainable business model. They prefer companies with established hardware and a proven track record, such as SpaceX
- Venture capital firms are broadening their focus to include defense technology. They are engaging advisors with military backgrounds to enhance their understanding of the complexities involved in defense investments
- Concerns remain about whether investors fully grasp the revenue potential of companies like Saronic. The ability to sell products to a diverse customer base is essential for long-term success
- Some investors express skepticism regarding Saronics valuation, suggesting it may exceed the companys current traction. The markets appetite for capital-intensive defense projects is still uncertain
- Cory Weinberg highlights the importance of thorough due diligence in defense technology investments. As more capital flows into this sector, understanding the underlying business models becomes increasingly critical
2100.0–2400.0
Encore is developing AI-native data infrastructure and collaborates with over 300 AI teams across various applications, including autonomous vehicles and surveillance systems. The company has raised $60 million and is valued at over half a unicorn, reflecting strong investor interest in its technology.
- Encore is building AI-native data infrastructure and collaborating with over 300 top AI teams. Their work spans various applications, including autonomous vehicles and surveillance systems
- The company has raised $60 million and is valued at over half a unicorn. This indicates significant investor interest in their innovative technology
- Good data in robotics is context-dependent. It requires diversity to avoid redundancy and ensure models can handle various scenarios and edge cases
- Incomplete data can lead to failures in AI systems. These systems lack the human intuition needed to generalize across different conditions and environments
- Humanoid robots are gaining popularity as many companies develop robots designed to integrate into human workflows. These robots are being used in homes, warehouses, and farms
- Encores services involve collecting data for robotic applications. This may include hiring teams or simulating environments to ensure comprehensive data coverage
2400.0–2700.0
Robotics requires customers to collect the right data for model training, which is essential for accurate world representation. The company has developed a scalable software platform to support data collection and deployment of robotic systems.
- Robotics lacks the luxury of pre-training on the internet, unlike language models. Customers must collect the right data for model training, which is crucial for building accurate representations of the world
- The process includes pre-training and post-training steps, such as data annotation and alignment. Software solutions help customers create a comprehensive robotics foundation model from the beginning
- Customers are starting to shift towards actual deployments of robotic systems, indicating rapid advancements in the field. Support is provided throughout the entire journey, from data collection to post-deployment operations
- Exception handling is a significant aspect of the software. It helps robots navigate errors during tasks, ensuring they can adapt and improve their performance over time
- Data collection requires both human operations and a robust software platform to be effective. A combination of these elements is necessary to cover the full data spectrum for physical AI applications
- The company has focused on building a scalable software platform capable of handling petabytes of data. This infrastructure will provide a competitive advantage as they ramp up their data collection warehouse
Robots Need Better Data
Full timeline
0.0–300.0
Robots often fail to adapt to minor environmental changes due to insufficient training data. Comprehensive data collection is essential for effective robot training to ensure performance in real-world situations.
- Robots often struggle with minor changes in their environment due to insufficient training data. For example, a robot trained in good lighting may fail if the lighting changes or if furniture is moved
- Humans can generalize their experiences and adapt to various conditions, while AI systems lack this intuition. This difference in adaptability can lead to failures in tasks that seem trivial to humans
- Training data must encompass a wide range of scenarios to be effective. Relying on just a few angles or shots is inadequate for teaching robots how to navigate different contexts
- Comprehensive data collection is essential for effective robot training. Full coverage of an environment ensures that robots can handle unexpected changes and variations in their surroundings
- The quality of training data directly impacts a robots performance in real-world situations. Without diverse and complete data, robots are likely to encounter challenges that humans would easily manage
- A robot may have extensive data from a well-lit room where it makes a bed. However, if conditions change, such as stormy weather or a different bed position, the robot may struggle because it lacks that specific context
The Data Behind Humanoid Robots
Full timeline
0.0–300.0
Encord is developing AI-native data infrastructure for over 300 AI teams, focusing on applications like autonomous vehicles and delivery drones. The company has raised $60 million and is valued at over half a unicorn, highlighting its growth potential in the robotics sector.
- Encord builds AI-native data infrastructure for over 300 top AI teams, focusing on applications such as autonomous cars and delivery drones
- The company has raised $60 million and is valued at over half a unicorn, indicating significant growth potential in the robotics sector
- Good data in robotics is context-dependent. Diversity is crucial to avoid redundancy and ensure comprehensive coverage of various scenarios
- Incomplete data can lead to failures in AI systems. These systems lack the intuition that humans possess to generalize across different conditions
- Humanoid robots are gaining traction. Many companies are developing robots designed to integrate into human workflows in homes, warehouses, and farms
- Encord assists customers in collecting the right data for their models. The company emphasizes the importance of tailored data collection over internet pre-training
300.0–600.0
Encord is developing a comprehensive software platform that supports the entire lifecycle of robotics deployment, from data collection to operational management. The company emphasizes the importance of combining software with human operations to meet the data needs of physical AI applications.
- Models trained on collected data learn how the world works and build representations of it. After data collection and pre-training, the focus shifts to post-training steps like annotation and alignment
- The software provided by Encord helps customers create a complete data flywheel from the beginning. This includes all necessary software and services to develop their robotics foundation model
- Customers are beginning to deploy their robots, which is an exciting development. Encord supports these deployments by assisting with the operational aspects of the robots once they are in use
- The company employs a team to manage the robots during their operations. They address any errors that occur, and this process, known as exception handling, is crucial for the softwares effectiveness
- Encord combines both software and human operations to ensure comprehensive data collection. This dual approach is essential for meeting the full data needs of physical AI applications
- The company has spent years developing a scalable software platform capable of handling petabytes of data. This infrastructure will provide a significant advantage as they ramp up their data collection efforts
Gemini 3.1 Just Dropped. SuperIntelligence Is Coming. We're Fine.
Full timeline
0.0–300.0
Gemini 3.1 has been released as an incremental upgrade in AI technology. Sam Altman from OpenAI claims that we may be only two years away from early versions of super intelligence, which could surpass current CEOs and scientists in capability.
- Gemini 3.1 has landed as an incremental upgrade
- Sam Altman from OpenAI claims we are only about two years away from super intelligence
- Dario Amote is the CEO of Anthropic
- There is tension between Sam Altman and Dario Amote, highlighted by their refusal to hold hands at the AI summit
- Sam Altman discussed the potential for super intelligence at the AI summit
- By the end of 2028, more of the worlds intellectual capacity could reside inside data centers than outside
- Super intelligence could outperform current CEOs and scientists
300.0–600.0
The discussion centers on the rapid advancements in AI technology, particularly the incremental improvements seen in models like Gemini 3.1. There is a notable divide in perception between those deeply involved in AI and the general public, with concerns about technological job loss and the implications of superintelligence.
- Incremental increases in AI models are coming faster and quicker
- There is a divide in perception of AI between those in the bubble and the general public
- Many people feel a sense of defeat regarding the rapid advancement of AI
- Technological job loss is a concern, with some hoping for superintelligence improvements
- Recent benchmarks show significant improvements in AI performance
- Gemini 3.1 achieved a score of 77% on the Arc AGI2 benchmark, up from 31.1% for Gemini 3
- User experience with AI is more important than benchmark scores
600.0–900.0
The discussion focuses on the advancements in AI technology, particularly the improvements in Gemini 3.1, which has significantly reduced hallucinations compared to its predecessor. Additionally, the conversation highlights the benefits of using paid AI services over free models and introduces Google's new Photoshoot feature for product photo manipulation.
- One of the watchers has a new book coming out called AI for cavemen
- Gemini 3.1 has a big reduction in hallucinations compared to Gemini 3.0
- Hallucinations in AI are improving but may never be completely solved
- Using paid AI services shows a massive difference in performance compared to free models
- The speaker is running four VPSs in the cloud instead of buying Mac minis
- Gemini 3.1 has made the speakers fleet of agents more efficient and intelligent
- Google rolled out an update to its Pompeii app service called Photoshoot
- Photoshoot allows easy manipulation of product photos for advertising and brand use cases
900.0–1200.0
The discussion revolves around the competitive landscape in AI, highlighting the dominance of large companies over smaller startups and the incremental updates in AI models like Sonnet 4.6 and Gemini 3.1. There are concerns about the implications of these advancements, particularly regarding user experience and the potential for dissatisfaction with deprecated models like GPT 4.0.
- Big companies are expected to dominate over smaller specialized startups
- Sonnet 4.6 is an incremental update that is cheaper than Opus 4.6
- Sonnet 4.6 is about 25% cheaper than Opus 4.6
- Sonnet 4.6 is close to Opus 4.6 in performance
- Sonnet 4.6 is noted for being a little over-eager in its operations
- Gemini 3.1 is currently the most impressive model for creating art using code
- There are rumors of a new mode called Citron mode coming to OpenAI
- GPT 4.0 has been depreciated, causing dissatisfaction among some users
1200.0–1500.0
The discussion addresses complaints about GPT 5.2 being overly focused on coding and bland, with anticipation for the upcoming GPT 5.3 release. Additionally, it highlights the introduction of new Patreon tiers to support AI tool development and the controversy surrounding C Dance 2.0, which has faced condemnation from major Hollywood studios.
- Complaints about GPT 5.2 being bland and focused too much on coding
- Anticipation for GPT 5.3 release within a week
- Introduction of new Patreon tiers at $10 and $25 a month
- Financial support from subscribers used to pay for AI tools
- C Dance 2.0 has been nerfed and is not officially out yet
- Hollywood studios including Disney, Paramount, and Netflix have condemned C Dance 2.0
- Issues with copyright restrictions on AI-generated content
- A video created by the Door Brothers claimed to make a $200 million AI movie in one day
1500.0–1800.0
The discussion focuses on the advancements in AI-generated content, particularly in film, emphasizing the need for creators to present a fully realized vision when pitching projects. AI tools like C.Dance 2.0 enable rapid production of video content that previously required extensive time and resources.
- The discussion revolves around the quality and consistency of AI-generated content, particularly in film
- The speaker emphasizes the need for creators to have a fully realized vision when pitching projects
- AI tools like C.Dance 2.0 allow for quick production of video content that previously took much longer
- The Door Brothers are noted for their work involving political figures in their narratives
- Charles Curran is mentioned for creating viral AI videos, including one that uses a famous Star Wars meme
- The ease of creating parody videos has significantly increased with AI technology
2100.0–2400.0
The discussion centers on the distinction between AI-generated music and traditional music, highlighting the speaker's experience creating songs across various genres using an AI tool. There are concerns about the quality of AI music, particularly regarding vocal production and the uncertainty of its target audience.
- The speaker believes its important to distinguish between AI-created music and non-AI music
- The speaker created four songs using the same prompt in different genres
- The prompt used was about McNuggets and their sweet and sour sauce
- The genres explored included emo, 90s rap, reggaeton, and folk
- The speaker noted that the AI music sounded better produced but still had an AI quality
- There is a desire for improvements in AI music tools, particularly in vocal quality
- The speaker expressed uncertainty about who would use the new AI music model
2400.0–2700.0
Mr. Tib's is an AI-powered assistant developed by Open Claw, which can be accessed through various communication methods.
- Mr. Tibs is an AI powered assistant powered by open claw
- Mr. Tibs can be summoned through telegram, phone call, or email
- The founder of open claw was hired by open AI
- Open claw is intended to remain open source despite the founders move
- Hermit claw allows for sandbox exploration
- Contra is a marketplace for agents to buy creative products
- Advancements in memory systems and best practices are being made in open claw
2700.0–3000.0
The discussion highlights advancements in robotics, particularly a recent television production in China featuring unitary robots performing kung fu. There is also mention of AI-generated videos that can convincingly mimic real-life actions, raising concerns about the blurring lines between reality and AI-generated content.
- Theres an avocado themed model that has not been revealed yet
- A large television production in China featured unitary robots performing kung fu
- The rehearsal video showed robots learning to cooperate in a concrete room
- The current robot performances are significantly better than last years
- Videos of robots with machine guns and drones are frequently seen on social media
- An AI-generated video of a Boston Dynamics robot dancing fooled many viewers
3000.0–3300.0
The discussion focuses on the potential of AI-generated content in filmmaking, highlighting a specific video that resembles 80s sci-fi films. It emphasizes the distinction between creative works produced by experienced creators and those generated by AI tools.
- Turning your agent into a complex software like Blender can yield interesting results
- AI creators are producing unique content, as demonstrated by Ryan Lightbornes video
- The video resembles an 80s sci-fi film, reminiscent of Ice Pirates
- Such creative works may not be produced by Hollywood anymore
- There is potential for niche audiences to support creative individuals in filmmaking
- Character creation and world building are essential aspects of filmmaking
- The output from experienced creators differs significantly from AI-generated content
The AI lab market map, Robinhood brings startups to retail, GLPs & hedge funds | Diet TBPN
Full timeline
0.0–300.0
Tyler Cosgrove created a comprehensive market map that includes every company, which has caused disappointment for a VCS associate who was working on a similar project. The map utilizes data from seven and a half million English Wikipedia articles, revealing clusters of companies in a 2D format.
- Tyler Cosgrove made a final market map that includes every company
- There was a VCS associate who was devastated because all the companies they planned to include are on Tylers market map
- Wikipedia is considered an underrated data source for information
- Tyler ran seven and a half million English Wikipedia articles through an embedding model
- Every Wikipedia article has a vector representation in a high-dimensional space
- The mapping down to 2D reveals clusters of companies, such as theater and space companies
- The appearance of the map resembles the United States but is random
- The map is interactive, allowing users to look up companies
- Neo Labs have exploded in recent times, with many founders featured on the show
- The term Neo Lab is broad and not clearly defined
300.0–600.0
The discussion centers on various AI labs, highlighting the distinction between traditional and neo labs, as well as their approaches to pre-training. Mr.
- They have their own base pre-trained
- Mr. Alkana has become a leader in European AI
- Coher is a Canadian company that has done its own pre-trains
- Legacy labs are entrenched in big enterprises
- Bell Labs was founded by Alexander Graham Bell
- Facebook AI research produced many OG research papers
- Priming tolect is a quintessential Neo lab
- Thinking machines and SSI set the tempo for research outside of big trad labs
- Trad SaaS labs use data inside big enterprises
- Neo SaaS labs are more startup-focused
600.0–900.0
Various types of AI labs are discussed, including post-lost labs that train models, safety labs focused on mechanistic interpretability, and consumer labs aimed at enhancing human interaction with models. The classification of these labs into categories such as visual, neo, and wet labs highlights their distinct research focuses and operational approaches.
- Post-lost lab trains models and works on top of those models
- Safety lab has a big safety team and focuses on mechanistic interpretability
- Eleuther AI is a similar kind of lab focused on research
- Consumer lab focuses on creating models that work better alongside people
- Visual labs produce video or images and are research-focused
- Neo lab focuses on a single moonshot idea
- Wet labs are biology-focused labs
900.0–1200.0
Robin Hood discusses the democratization of private market investments, allowing individuals access to companies like Databricks and Revolut. Concerns are raised about the fund's structure potentially leading to significant divergence from net asset value.
- Robin Hood says historically investing in private markets was limited to institutions and the elite, but not anymore
- You can now get exposure to private companies like Databricks, Mercore, Revolut, Airwallix, Boom, Supersonic, Ramp, ORA, and Stripe
- Databricks was bought at $150 per share, now trading at $204
- Ramp was bought at $90, now trading at $98
- Airwallix was bought at $21, now trading at $18.8
- Mercore was bought at $7.14, now trading
- The structure of this fund is broken, leading to potential divergence from net asset value
- Elon Musk announced that XAI is moving away from traditional academic benchmarks
1200.0–1500.0
The discussion revolves around the competitive landscape of space exploration, emphasizing the need for multiple launch companies to enhance capabilities against adversaries. Additionally, a nonprofit initiative led by Toby Rice aims to improve energy access in impoverished countries through a diverse range of energy solutions.
- A lot of people have viewed it as a warning shot to Elon Musk focused on SpaceX going to Mars
- The US needs two launch companies competing vigorously against each other
- Healthy competition is necessary for the US to move quickly against adversaries
- The moon race is shaping up well, with a focus on who could get there in 2028
- Toby Rice is starting a nonprofit to tackle a lack of access to modern energy infrastructure in poor countries
- Energy Corps sees a role for a broader spectrum of solutions from fossil fuels to solar panels and nuclear plants
- The Rockefeller Foundation has endorsed the approach of Energy Corps
1500.0–1800.0
The discussion centers on Brazil's potential decision to clear the Amazon rainforest for industrialization, which raises environmental concerns. Additionally, the conversation explores the future of humanoid robots, with claims of their ability to construct large-scale projects rapidly.
- Brazils potential clearing of the Amazon rainforest for industrialization is debated
- David Holmes claims 5 million humanoid robots could build Manhattan in six months
- The discussion includes the future of 10 billion humanoid robots by 2045
- Richard mentions a future where everything will be blueberries
- Orren Hoffman discusses the negative impact of ozempik on business decisions
- Dr. Cameron Maximus suggests a microdose of Terzepetide and a macrodose of testosterone to increase drive
- A VC advises to find an alleyway to hide from improving models like Open AI and Anthropic
Physical AI Could Be a Trillion Dollar Market by 2035
Is Elon Musk's robotics vision too far-fetched? Plus, the advantages of industrial robots
Top AI Scientist: High-Paying Jobs AI Can't Replace in 2026 (And How to Get Them) | Daniela Rus
Full timeline
0.0–300.0
The discussion focuses on the importance of training in AI and robotics for individuals in repetitive jobs. It emphasizes the collaboration between humans and machines, highlighting that while AI will enhance cognitive tasks, robots will assist with physical tasks.
- For someone doing a repetitive job, they should be training in AI and robotics
- AI will support cognitive aspects of jobs, while robots will support physical aspects
- People will not lose jobs to AI, but to others who know how to use AI effectively
- It is important to keep learning and stay current with applicable tools in ones field
- The future will involve hybrid teams of humans and robots working together
- This synergy can free people from routine work, allowing focus on strategic aspects
- Edge AI is already here and has the potential to be more widely utilized
300.0–600.0
The discussion centers on the transformative potential of AI, particularly through on-device applications that could democratize innovation and reduce costs. By 2030, there is an expectation of increased integration of robots in both service industries and homes, although challenges remain for in-home robotics.
- AI is bringing most value through industrial installations that are huge and costly
- AI could democratize innovation by moving on-device, similar to the transition from mainframe computers to PCs
- On-device AI would be cheaper and more private compared to cloud interactions
- The development of AI tools could enable individuals to build startups without extensive developer support
- By 2030, there could be robots in homes, such as a robot garbage can and humanoid robots offering assistance
- The path from successful research experiments to full-blown products takes a long time
- By 2030, the service field may see a lot of robots, while in-home robotics will be more challenging
600.0–900.0
The discussion focuses on advancements in AI and robotics, particularly in content creation and audio production. It highlights the current limitations of humanoid robots in household tasks and the need for improved AI capabilities.
- Editing that once took days now takes hours
- Short form videos are created in minutes
- Scripts are written with AI support
- AI platforms handle most routine work in audio production
- labs Studio 3.0 has changed production workflow
- Voiceovers can be generated with exact emotional tone
- Content can be instantly dubbed in 20 plus languages using voice clones
- Humanoid robots are still far from being effective household helpers
- AI tools today do not have common sense
- Humanoid robots can learn tasks from fewer data examples
900.0–1200.0
The discussion addresses the lack of workforce support for elder care and the potential of robots like SoftMimic to assist with simple tasks. It emphasizes the necessity of technological and AI literacy for everyone, alongside fostering curiosity and critical thinking in education.
- We dont have the workforce to support the needs of aging
- There are simple tasks in elder care like getting out of bed that lack tools and equipment
- SoftMimic is a system designed to teach human walking
- The robot must handle unexpected contacts with the environment
- SoftMimic trains the robot to imitate human motions and respond to external forces
- Technological literacy, including AI literacy, is essential for everyone
- Not everyone needs to be an AI expert, but everyone should know something about it
- An inclusive general education is important for understanding our world and future
- Qualities like curiosity, creativity, and critical thinking are important to teach children
1200.0–1500.0
The discussion emphasizes the importance of formal education in developing critical thinking and problem-solving skills, while also highlighting the role of knowledge in fostering creativity. It expresses a desire for robots to be seamlessly integrated into daily life and for breakthroughs in healthy longevity.
- I take all the tips from these podcasts and apply them to my personal life, to my investment portfolio and to my businesses
- Formal university education is very important because at university you learn how to think, solve problems, and find your way forward
- Knowing things enables us to be creative and connect seemingly disparate concepts
- Creativity is about looking at the world in different ways and connecting parts of the world that are seemingly different
- I would like to see robots integrated into the fabric of life, making them useful, capable, trustworthy, and reliable
- Advances in hardware, algorithms, and interaction with machines are continuous challenges in robotics
- Healthy longevity and possibly reversing aging is a breakthrough I would love to see in my lifetime
- Life is different from what we imagine; one must be flexible, adaptive, and open-minded
HAI Seminar: Human Skill Augmentation in Robot-Assisted Surgery
Full timeline
0.0–300.0
The shortage of surgical procedures leads to significant medical errors, which are a major cause of death, prompting the need for augmented systems in robot-assisted surgery.
- There are around 230 million procedures conducted worldwide
- The world is short of at least 140 million surgical procedures
- Medical errors are the third leading cause of death in the United States
- Around a quarter of a million Americans die each year due to medical errors
- % of medical errors occur inside the operating room
- % of surgical complications from medical errors are avoidable
- The research focuses on designing systems to augment humans in robot assisted surgery
- One aspect of the research is improving motor skill learning in surgery
- Another aspect involves designing autonomous systems for repetitive tasks like suturing
- Robot assisted surgery is a sub-paradigm of minimally invasive surgery
300.0–600.0
Robots possess unique capabilities that allow them to perform multiple tasks simultaneously, enhancing surgical precision and situational awareness for surgeons.
- Robots have unique capabilities that are fundamentally different than humans
- Most systems in the market have only one robotic manipulator holding one camera for the surgeon
- Robots can have more than one pair of eyes, providing better situational awareness
- Robots can focus on multiple locations simultaneously without cognitive overload
- Typical surgical systems like the da Vinci have four robotic manipulators
- Robotic manipulators are singularly controlled by the human surgeon through a switching control system
- An example of robotic use is holding an ultrasound scan to locate a tumor inside the kidney
600.0–900.0
The automation of an extra robotic arm enhances surgical precision by allowing autonomous assistance, reducing the risk of human error during operations.
- The surgeon can accidentally move robots outside the field of view
- The goal is to automate an extra robotic arm to assist the surgeon
- The third or fourth arm can be autonomous, helping the surgeon
- Test cases for the robotic arm fall into two categories: physical interaction and non-physical interaction
- The design of the robotic arm must be force aware to handle different tissue stiffness levels
- Applying too little force may prevent proper grasping, while too much force can tear tissue
- The research focuses on an autonomous tissue retraction test
- A six degree freedom force torque sensor measures forces and torques during interaction
- Data collected includes kinematic data, vision data, and force data
- Imitation learning is used to digitize surgery and predict robotic arm movements
900.0–1200.0
The force policy demonstrates superior performance by applying significantly less force and achieving higher success rates in surgical tasks, leading to safer and more effective procedures.
- Collected 60 demonstrations over one hour using the da Vinci surgical system
- Trained two policies: a force policy and a no force policy
- The force policy is three times more successful than the no force policy on 50 rollouts
- The force policy applied 62% less force compared to the no force policy
- The force policy applied smaller values of forces, typically less than one Newton, more often
- The force policy was more gentle over time in its interaction with the tissue
- The force policy was 3.5 times more successful on unseen tissue samples compared to the no force policy
- The force policy applied 110% less force on unseen tissue samples compared to the no force policy
1200.0–1500.0
The updated teleoperation system enables two humans to control three robotic arms simultaneously, enhancing surgical precision and collaboration.
- The extrobotic arm can work with teleoperated arm surgeons surgeons for tri-lateral manipulation
- A data collection system was updated to allow two humans to control three robotic arms simultaneously
- One human controls two arms with two hands, while a second human uses the Phantom Omni device device to control the third arm
- The architecture uses the action-tracking transformer model with kinematic and vision data as inputs
- The system can predict the motion of one robotic arm autonomously while the others are teleoperated
- The project started with a two-handed task of object handover and progressed to a trial-literal task with three arms
- Demonstrations showed various configurations of autonomous and teleoperated arms working together
1500.0–1800.0
Robots can operate in parallel at multiple locations without cognitive limitations, enhancing efficiency in delicate tasks like suturing.
- Lift arm is teleoperated, right arm is autonomous
- Successful handover of objects
- Middle arm is teleoperated, two side arms are autonomous
- Goal is to pick Z3 objects and place them into a common location
- ACT architecture could be used for three handed tests
- Three handed tests can be collaborative
- Ongoing work involves collaborative analysis of automation and teleoperation
- Robots can focus on two different locations simultaneously
- Humans find it difficult to focus on two locations at the same time
- Robots do not have cognitive limitations like humans
1800.0–2100.0
The hierarchical state machine execution model enhances surgical robotics by enabling concurrent operations, resulting in nearly double the speed of traditional methods.
- We simply set the execution as a way the autonomous execution is happening using a state machine
- It allows having state machines and sub-state machines and hence the word hierarchical
- Our proposed execution model was almost twice as fast as the other two execution models
- By having multiple arms, we open up surgical robotics as one application area for multi-robot systems
- Robots can have more than one pair of eyes
- We designed an extra camera for the surgeon that can provide an auxiliary view
- The design of the camera is a cylinder inserted vertically through the same incisions
2100.0–2400.0
The dual view system in robotic surgery enhances accuracy and speed in surgical training, leading to improved surgeon skills.
- We can have a multi camera multi view system in robot assist surgery
- Conducted a user study comparing single view group and dual view group
- The goal of the surgical training test called Why Richey or Test is to move a ring from one side of the rail to the other
- The dual view group had a picture in picture view
- The dual view group was 35% more accurate in the training test
- The dual view group was 25% faster in performing the test
- The design of the pickup camera allows for adjustable inter-popularly distance
- Increasing the distance between vision sensors improves perception of the scene
2400.0–2700.0
The use of 20 millimeters and 30 millimeters in the dip-thranking test resulted in lower Hamming distance scores, indicating better performance in sequencing tasks.
- Subjects performed a dip-thranking test to sequence poles from shortest to tallest
- The required precision for the task was 2.5 millimeters
- The sequences provided by subjects were compared to a ground truth sequence using Hamming distance
- Lower Hamming distance scores indicated better performance
- millimeters was identified as the sweet spot for optimal performance
- The results suggest a shift in the design of endoscopes and cameras
- The idea of a dynamic or adaptive camera baseline was proposed
- The current camera system is teleoperated by the surgeon, with ongoing work to automate it
- Surveys indicate that the optimal forces used by surgeons vary depending on the task
2700.0–3000.0
The integration of robotics in surgery aims to enhance human capabilities, fostering trust among surgeons and patients, which is crucial for adoption.
- Did we do like error analysis before we jump into sub projects?
- Some of this was more opportunistic because I find out that, you know, when you, theres also a beauty to tell surgeons, okay, heres another cool technology
- Action shot transformers for example is an obvious case from a technical perspective
- I think now is the time to do the human augmentation
- We want to gain the surgeons trust
- The way to gain the trust of a critical mass of the surgical community is to do things like, you know, automating the camera for them
- Patients actually just because of, you know, what they hear in the media, they would actually ask if they can be operated on using Zorab
3000.0–3300.0
The development of autonomous surgery technology aims to improve patient outcomes, which is essential for gaining patient trust.
- Researchers have the responsibility to be forthcoming about the capabilities of autonomous surgery technology
- Gaining patient trust is linked to achieving great patient outcomes
- Demonstrating improved patient outcomes can help build trust in new surgical technologies
- There is a need to explore what tasks in surgery should be automated
- Current literature focuses on a limited number of tasks for automation in surgery
- Engaging with surgeons over time can help identify opportunities for automation
- The potential of a data-driven approach in robot-assisted surgery is an exciting area of research
- The Albin H. Embodiment Initiative involves collaboration between Nvidia, Johns Hopkins, and a technical university clinic
- The initiative aims to collect extensive data from various levels of surgery