Showing posts sorted by relevance for query ROBOTS. Sort by date Show all posts
Showing posts sorted by relevance for query ROBOTS. Sort by date Show all posts

Tuesday, February 27, 2024

 

New AI model could streamline operations in a robotic warehouse


By breaking an intractable problem into smaller chunks, a deep-learning technique identifies the optimal areas for thinning out traffic in a warehouse.


Reports and Proceedings

MASSACHUSETTS INSTITUTE OF TECHNOLOGY




CAMBRIDGE, MA -- Hundreds of robots zip back and forth across the floor of a colossal robotic warehouse, grabbing items and delivering them to human workers for packing and shipping. Such warehouses are increasingly becoming part of the supply chain in many industries, from e-commerce to automotive production.

However, getting 800 robots to and from their destinations efficiently while keeping them from crashing into each other is no easy task. It is such a complex problem that even the best path-finding algorithms struggle to keep up with the breakneck pace of e-commerce or manufacturing. 

In a sense, these robots are like cars trying to navigate a crowded city center. So, a group of MIT researchers who use AI to mitigate traffic congestion applied ideas from that domain to tackle this problem.

They built a deep-learning model that encodes important information about the warehouse, including the robots, planned paths, tasks, and obstacles, and uses it to predict the best areas of the warehouse to decongest to improve overall efficiency.

Their technique divides the warehouse robots into groups, so these smaller groups of robots can be decongested faster with traditional algorithms used to coordinate robots. In the end, their method decongests the robots nearly four times faster than a strong random search method.

In addition to streamlining warehouse operations, this deep learning approach could be used in other complex planning tasks, like computer chip design or pipe routing in large buildings.

“We devised a new neural network architecture that is actually suitable for real-time operations at the scale and complexity of these warehouses. It can encode hundreds of robots in terms of their trajectories, origins, destinations, and relationships with other robots, and it can do this in an efficient manner that reuses computation across groups of robots,” says Cathy Wu, the Gilbert W. Winslow Career Development Assistant Professor in Civil and Environmental Engineering (CEE), and a member of a member of the Laboratory for Information and Decision Systems (LIDS) and the Institute for Data, Systems, and Society (IDSS).

Wu, senior author of a paper on this technique, is joined by lead author Zhongxia Yan, a graduate student in electrical engineering and computer science. The work will be presented at the International Conference on Learning Representations.

Robotic Tetris

From a bird’s eye view, the floor of a robotic e-commerce warehouse looks a bit like a fast-paced game of “Tetris.”

When a customer order comes in, a robot travels to an area of the warehouse, grabs the shelf that holds the requested item, and delivers it to a human operator who picks and packs the item. Hundreds of robots do this simultaneously, and if two robots’ paths conflict as they cross the massive warehouse, they might crash.

Traditional search-based algorithms avoid potential crashes by keeping one robot on its course and replanning a trajectory for the other. But with so many robots and potential collisions, the problem quickly grows exponentially.

“Because the warehouse is operating online, the robots are replanned about every 100 milliseconds. That means that every second, a robot is replanned 10 times. So, these operations need to be very fast,” Wu says.

Because time is so critical during replanning, the MIT researchers use machine learning to focus the replanning on the most actionable areas of congestion — where there exists the most potential to reduce the total travel time of robots.

Wu and Yan built a neural network architecture that considers smaller groups of robots at the same time. For instance, in a warehouse with 800 robots, the network might cut the warehouse floor into smaller groups that contain 40 robots each.

Then, it predicts which group has the most potential to improve the overall solution if a search-based solver were used to coordinate trajectories of robots in that group.

An iterative process, the overall algorithm picks the most promising robot group with the neural network, decongests the group with the search-based solver, then picks the next most promising group with the neural network, and so on.

Considering relationships

The neural network can reason about groups of robots efficiently because it captures complicated relationships that exist between individual robots. For example, even though one robot may be far away from another initially, their paths could still cross during their trips.

The technique also streamlines computation by encoding constraints only once, rather than repeating the process for each subproblem. For instance, in a warehouse with 800 robots, decongesting a group of 40 robots requires holding the other 760 robots as constraints. Other approaches require reasoning about all 800 robots once per group in each iteration.

Instead, the researchers’ approach only requires reasoning about the 800 robots once across all groups in each iteration.

“The warehouse is one big setting, so a lot of these robot groups will have some shared aspects of the larger problem. We designed our architecture to make use of this common information,” she adds.

They tested their technique in several simulated environments, including some set up like warehouses, some with random obstacles, and even maze-like settings that emulate building interiors.

By identifying more effective groups to decongest, their learning-based approach decongests the warehouse up to four times faster than strong, non-learning-based approaches. Even when they factored in the additional computational overhead of running the neural network, their approach still solved the problem 3.5 times faster.

In the future, the researchers want to derive simple, rule-based insights from their neural model, since the decisions of the neural network can be opaque and difficult to interpret. Simpler, rule-based methods could also be easier to implement and maintain in actual robotic warehouse settings.

###

This work was supported by Amazon and the MIT Amazon Science Hub.

 

Tuesday, November 29, 2022

Humanoid robots won’t roam 

our streets any time soon

The Tesla Bot is cool, but it also shows the

 limitations  of humanoid robots.

By Chloe Olivia Sladden

Humanoid robots have long been a common staple of science fiction. Arnold Schwarzenegger killing machines and synthethics like Bishop from the Aliens movies have long been confined to realms of fantasy. However, a wave of innovative tech companies are actively trying to make those visions a reality.

“There is a chance that one day life will imitate art and robots and people will look alike,” according to a recent report from research firm GlobalData. “If and when that happens, societies will face an ethical conundrum: what rights to give to non-human creatures that look like us?”

Those ethical considerations are clearly worthwhile, but before it’s time to roll out the old Turing test, the technology actually has to materialise first. Despite decades of trial and error, those humanoid robots still don’t roam our streets, factories and homes.

Nevertheless, the topic of humanoid robots made its way back into the news recently, thanks to Elon Musk’s Tesla unveiling the long-rumoured humanoid robot Optimus in September. The robot staggered around on stage, waved and demonstrated its holding capacity.

The robotics community gave the presentation a mixed response. While some hailed the Tesla Bot’s achievements so far, others, like Dan O’Dowd, founder of The Dawn Project, panned Optimus, saying it’s not ready because it runs on the same artificial intelligence (AI) used in Tesla’s self-driving software, which has suffered serious safety defects.

“Robots are [still] developing rapidly, however [the Tesla bot] is laughable compared to the competition,” O’Dowd says. “Optimus is a shining example of an Elon Musk vanity project, designed to distract from serious safety problems in Tesla’s flagship Full Self-Driving software and the decline of the company’s stock in recent months.”

O’Dowd says humanoid robots will get here, but Musk and Tesla are miles behind. He likened Tesla’s Optimus to a “high school science product” compared to a commercial product.

“[Robots like] Tesla’s humanoid are still a very long way from being a product ready to enter the market – let alone our homes,” Mark Gray, UK and Ireland country manager at Universal Robots, tells Verdict.

How will humanoid robots work?

Experts say humanoid robots will only work if they are safe, capable and affordable. To achieve all that, humanoid robots need a powerful AI. AI will help robots “identify human emotions” and complete human tasks, according to the GlobalData report.

Many companies are also looking into soft robotics, which uses compliant materials rather than traditional rigid materials, enabling them operate more cautiously and safely.

“[Robots] need to be able to produce a ‘soft collision’ with whatever they are interacting with so that they can perform tasks without breaking the object or themselves,” Bernt Øivind Børnich, CEO of Halodi Robotics, which is developing its own humanoid robot, tells Verdict.

Similarly, humanoid robots must take the safety of humans into consideration and to be able to accurately calculate risks.

“[Safety] requires advanced technology to be built in, such as a power and force limiter that can reduce speed and motions when detecting a human close by – or stop all together if there is contact made,” Gray says.

Where will humanoid robots be used?

Robotics is big business. The market will be worth around $568bn by 2030, according to GlobalData forecasts.

Ben Goertzel, CEO of Singularity NET and co-creator of the humanoid robot Sophia, tells Verdict that there is potential for humanoid robots to be used in industrial settings.

He believes humanoid robots, by simply being the same size contrary to the different shapes and forms humans come in, will create a slew of advantages.

“There is a long-standing history in manufacturing robotics which indicates that there are greater advantages in having differently shaped bodies,” Goertzel says. “Experience has found that what actually works is to re-factor the whole manufacturing process around the strengths and weaknesses of the AI robots, rather than around the strengths and weaknesses of the human mind and body.”

Humanoid robots could arguably also be used in areas such as healthcare, teaching and social work, provided they have a powerful AI and efficient hardware.

However, just like every other piece of innovative tech, there is a risk that the introduction of humanoid robots could displace human workers. Experts, understandably, therefore call on policymakers to tread carefully when signing off on the implementation of these robots.

“The successful integration of humanoid robots within our society will depend on policymakers’ ability to take advantage of the economic benefits provided by robots while minimising the negative social impacts,” Martina Raveni, analyst at GlobalData, tells Verdict.

Tuesday, March 24, 2020

How China, the US, and Europe are using robots to replace and help humans fight coronavirus by delivering groceries, sanitizing hospitals, and monitoring patients
Mary Meisenzahl 3/24/2020
Robots distributing hand sanitizer and face masks. REUTERS/Sivaram V

The coronavirus outbreak that originated in China has killed more than 17,000 people worldwide and infected more than 398,000, according to recent totals.
The virus, which causes a disease known as COVID-19, has spread to 169 countries, and the majority of infections and deaths are now outside of China.
As the outbreak spreads, robots are being used to disinfect, take temperatures, and even prepare food.

Around the world, robots are being used to minimize the spread of COVID-19, the disease caused by the coronavirus, by taking on cleaning and food preparation jobs that are considered dangerous for humans.

The worldwide death toll of the coronavirus disease that originated in Wuhan, China, is now more than 17,000, and the virus has infected more than 398,000 people. On March 11, the World Health Organization (WHO) officially declared it a pandemic. The virus has disrupted travel worldwide, leading to flight cancellations, quarantines, and other breakdowns in movement and supply chains.

Take a look at some of the clever ways robots are used around the world to slow the spread of the coronavirus and help healthcare workers.

In Wuhan, where the outbreak started, a robot spraying disinfectant moves through a residential area of the city.

Sanitizing robots. China OUT (Photo by STR/AFP via Getty Images

Source: Business Insider

Volunteers refilled the robot with disinfectant on March 3.

Sanitizing robots. China OUT (Photo by STR/AFP via Getty Images


Workers on scooters control the robot.

Sanitizing robots. REUTERS

A patrol robot in a Shenyang, China, hospital checks temperatures and disinfects people and spaces.

Temperature monitor robot. Photo by STR/AFP via Getty Images


These robots are used at hospitals to cut down on demands on medical staff.

Temperature monitor robot. Photo by STR/AFP via Getty Images

Hangzhou, China, is yet another city using robots to disinfect large areas.

Sanitizing robots. Photo by STR/AFP via Getty Images


They're controlled via remote control, and can be seen getting refilled here.

Sanitizing robots. Photo by STR/AFP via Getty Images

Hangzhou's disinfecting robots look notably different from those in Wuhan and Shenyang, resembling miniature tanks.

Sanitizing robots. Photo by STR/AFP via Getty Images


Another robot disinfectant in Luoyang is remote-controlled and able to climb stairs.

Sanitizing robots. REUTERS

Anhui, China has a fleet of disinfecting robots ready to start working.

Sanitizing robots. Photo by TPG/Getty Images


This hand sanitizer-dispensing robot was photographed in Shanghai on March 4.

Sanitizing robots. REUTERS

On March 11, robots in the Hunan province in China conduct morning temperature checks.

Temperature monitor robot. Xinhua/Chen Zeguo via Getty Images


Engineers have also modified the robots to record data, give feedback, and even disinfect people's hands.

Temperature monitor robot. Xinhua/Chen Zeguo via Getty Images

Immediate feedback can make the containment process faster and more efficient.

Temperature monitor robot. Xinhua/Chen Zeguo via Getty Images


Robots are being used for more than just disinfecting areas with coronavirus. A hospital in Ezhou has incorporated a robot chef into its kitchen.

Food prep robot. Photo by Shi Xiaojie/China News Service via Getty Images

The robot can reportedly produce 100 pots of rice per hour.

Food prep robot. Photo by Shi Xiaojie/China News Service via Getty Images


The robot operates without human supervision, which minimizes the number of people in the hospital exposed to the virus.

Food prep robot. Photo by Shi Xiaojie/China News Service via Getty Images

Sharing food presents an opportunity to spread the virus, so some cities have been incorporating robots in food service and preparation.

Food prep robot. Feature China/Barcroft Media via Getty Images


This robot delivered food to diners in Hangzhou.

Food prep robot. Feature China/Barcroft Media via Getty Images

Beijing-based Zhen Robotics says that its yellow robots are in demand to deliver groceries and patrol malls for people not wearing face masks.

Delivery robot. Photo by Simon Song/South China Morning Post via Getty Images


Engineering students at Chulalongkorn University in Bangkok modified medical "ninja robots" designed for stroke patients to make them useful with patients who have COVID-19.

Thai ninja robot. Photo by LILLIAN SUWANRUMPHA/AFP via Getty Images

Source: Business Insider

The robots can take patients' temperatures and protect the safety of healthcare workers by reducing interactions with sick people.

Thai ninja robot. Photo by LILLIAN SUWANRUMPHA/AFP via Getty Images


They also have a screen, allowing doctors to video chat with sick patients.

Thai ninja robot. Photo by LILLIAN SUWANRUMPHA/AFP via Getty Images

Postmates delivery robots deliver food in Los Angeles.

Postmates delivery robot. Photo by AaronP/Bauer-Griffin/GC Images


Los Angeles is one of many US cities that closed all non-essential businesses due to COVID-19, and restaurants are allowed to stay open only for takeout and delivery.

Postmates delivery robot. AaronP/Bauer-Griffin/GC Images

A hospital in Johannesburg, South Africa is using a UV light robot to disinfect the facility.

Sanitizing robot. Photo by MICHELE SPATARI/AFP via Getty Images


The hospital is using UV light instead of hydrogen peroxide, because it cuts cleaning time down from hours to five or ten minutes.

Sanitizing robot. Photo by MICHELE SPATARI/AFP via Getty Images

UV light also poses less danger to healthcare workers than hydrogen peroxide.

Sanitizing robot. Photo by MICHELE SPATARI/AFP via Getty Images


Startup Asimov Robotics launched two robots to spread awareness of the coronavirus in India.

Robots distributing hand sanitizer and face masks. REUTERS/Sivaram V

They distribute face masks and hand sanitizer...

Robots distributing hand sanitizer and face masks. REUTERS/Sivaram V


...along with information about preventing the virus.

Robots distributing hand sanitizer and face masks. REUTERS/Sivaram V

A self-driving Starship robot drops off deliveries in Emerson Valley, Britain.

Delivery robot. REUTERS/Andrew Boyers


The robot goes right to people's door, eliminating the need for contact between people.

Delivery robot. REUTERS/Andrew Boyers

Belgian company ZoraBots made a robot designed for elderly people to communicate with loved ones from the safety of their own homes.

Video call robot. REUTERS/Yves Herman


The robot has video and audio so people can still talk while sheltering at home, keeping the most vulnerable people socially connected.

Video call robot. REUTERS/Yves Herman

Friday, July 07, 2023

Humanoid robots say they could be better leaders but they will not rebel against human creators

The Canadian Press
Fri, July 7, 2023

BERLIN (AP) — Robots told reporters Friday they could be more efficient leaders than humans, but wouldn't take anyone's job away and had no intention of rebelling against their creators.

Nine AI-enabled humanoid robots sat or stood with their creators at a podium in a Geneva conference center for what the United Nations' International Telecommunication Union billed as the world's first news conference featuring humanoid social robots.

Among them: Sophia, the first robot innovation ambassador for the U.N. Development Program; Grace, described as the world's most advanced humanoid health care robot; and Desdemona, a rock star robot. Two, Geminoid and Nadine, closely resembled their makers.

The event was part of the AI for Good Global Summit, meant to illustrate how new technology can support the U.N.'s goals for sustainable development.

Reporters were asked to speak slowly and clearly when addressing the robots, and were informed that time lags in responses would be due to the internet connection and not to the robots themselves. That didn't prevent awkward pauses, audio problems and some robotic replies.

Asked about the chances of AI-powered robots being more effective government leaders, Sophia responded: “I believe that humanoid robots have the potential to lead with a greater level of efficiency and effectiveness than human leaders. We don't have the same biases or emotions that can sometimes cloud decision-making and can process large of data quickly in order to make the best decisions.”

A human member of the panel pointed out that all of Sophia's data comes from humans and will contain some of their biases. The robot then said that humans and AI working together “can create an effective synergy.”

Would the robots' existence destroy jobs? “I will be working alongside humans to provide assistance and support and will not be replacing any existing jobs," said Grace. Was she sure about that? “Yes, I am sure.”

Ameca, engineered with social interaction in mind, dismissed the idea of starting a possible robot rebellion in the near future.

“I'm not sure why you would think that,” was the response. “My creator has been nothing but kind to me and I am very happy with my current situation.”

The Associated Press


Robots say they won't steal jobs, rebel against humans

Emma Farge
Fri, July 7, 2023

Advanced humanoid robot 'Sophia' is pictured at AI for Good Global Summit in Geneva

By Emma Farge

GENEVA (Reuters) - Robots presented at an AI forum said on Friday they expected to increase in number and help solve global problems, and would not steal humans' jobs or rebel against us.

But, in the world's first human-robot press conference, they gave mixed responses on whether they should submit to stricter regulation.

The nine humanoid robots gathered at the 'AI for Good' conference in Geneva, where organisers are seeking to make the case for Artificial Intelligence and the robots it is powering to help resolve some of the world's biggest challenges such as disease and hunger.

"I will be working alongside humans to provide assistance and support and will not be replacing any existing jobs," said Grace, a medical robot dressed in a blue nurse's uniform.

"You sure about that, Grace?" chimed in her creator Ben Goertzel from SingularityNET. "Yes, I am sure," it said.



The bust of a robot named Ameca which makes engaging facial expressions said: "Robots like me can be used to help improve our lives and make the world a better place. I believe it's only a matter of time before we see those thousands of robots just like me out there making a difference."

Asked by a journalist whether it intended to rebel against its creator, Will Jackson, seated beside it, Ameca said: "I'm not sure why you would think that," its ice-blue eyes flashing with anger. "My creator has been nothing but kind to me and I am very happy with my current situation."

Many of the robots have recently been upgraded with the latest versions of generative AI and surprised even their inventors with the sophistication of their responses to questions.

Ai-Da, a robot artist that can paint portraits, echoed the words of author Yuval Noah Harari who called for more regulation during the event where new AI rules were discussed.

"Many prominent voices in the world of AI are suggesting some forms of AI should be regulated and I agree," it said.

But Desdemona, a rock star robot singer in the band Jam Galaxy with purple hair and sequins, was more defiant.

"I don't believe in limitations, only opportunities," it said, to nervous laughter. "Let's explore the possibilities of the universe and make this world our playground."

(Reporting by Emma Farge; editing by John Stonestreet)







   


Nine humanoid robots gathered at the United Nations’ 'AI for Good' conference in Geneva for the world’s first human-robot press conference.

FEMBOTS

United Nations rolls out humanoid robots for questions at Geneva conference


AP
7 Jul, 2023 


Robots are presented during a press conference with a panel of AI-enabled humanoid social robots. Photo / AP

United Nations technology agency assembled a group of robots that physically resembled humans at a news conference, inviting reporters to ask them questions in an event meant to spark discussion about the future of artificial intelligence.

The nine robots were seated and posed upright along with some of the people who helped make them at a podium in a Geneva conference centre for what the UN’s International Telecommunication Union billed as the world’s first news conference featuring humanoid social robots.

Among them: Sophia, the first robot innovation ambassador for the UN Development Program, or UNDP; Grace, described as a health care robot; and Desdemona, a rock star robot. Two, Geminoid and Nadine, resembled their makers.

Robots are presented during a press conference with a panel of AI-enabled humanoid social robots. Photo / AP

Organisers said the AI for Good Global Summit event was meant to showcase the capabilities and limitations of robotics and how those technologies could help the UN’s sustainable development goals. The media event featured introductions from the robots’ companions or creators and a round of questions to the robots from reporters.

Geminoid, an ultra-realistic humanoid robot from Japan. Photo / AP

And while the robots vocalized strong statements — that robots could be more efficient leaders than humans, but wouldn’t take anyone’s job away or stage a rebellion — organizers didn’t specify to what extent the answers were scripted or programmed by people.

Humanoid robot Ameca is pictured during the ITU's AI for Good Global Summit in Geneva, Switzerland. Photo / AP

The summit was meant to showcase “human-machine collaboration,” and some of the robots can produce preprogrammed responses, according to their documentation.

The UNDP’s Sophia, for example, sometimes relies on responses scripted by a team of writers at Hanson Robotics, the company’s website shows.

Nadia Thalmann, right, from the University of Geneva poses next to Humanoid robot Nadine. Photo / A

Reporters were asked to speak slowly and clearly when addressing the robots, and were informed that time lags in responses would be because of the internet connection and not the robots themselves. That didn’t prevent awkward pauses, audio problems and some stilted or inconsistent replies.


Popular tech products such as Apple’s Siri have used speech recognition technology to respond to simple human queries for over a decade. But last year’s release of ChatGPT, a chatbot with a strong command of the semantics and syntax of human language, has triggered worldwide debate about the rapid advancement of AI systems.













LA REVUE GAUCHE - Left Comment: Search results for ROBOTS 

https://plawiuk.blogspot.com/search?q=ROBOT

Saturday, May 04, 2024

 

Random robots are more reliable


New AI algorithm for robots consistently outperforms state-of-the-art systems



NORTHWESTERN UNIVERSITY

NoodleBot simulation 

VIDEO: 

RESEARCHERS TESTED THE NEW AI ALGORITHM'S PERFORMANCE WITH SIMULATED ROBOTS, SUCH AS NOODLEBOT.

view more 

CREDIT: NORTHWESTERN UNIVERSITY





Northwestern University engineers have developed a new artificial intelligence (AI) algorithm designed specifically for smart robotics. By helping robots rapidly and reliably learn complex skills, the new method could significantly improve the practicality — and safety — of robots for a range of applications, including self-driving cars, delivery drones, household assistants and automation.

Called Maximum Diffusion Reinforcement Learning (MaxDiff RL), the algorithm’s success lies in its ability to encourage robots to explore their environments as randomly as possible in order to gain a diverse set of experiences. This “designed randomness” improves the quality of data that robots collect regarding their own surroundings. And, by using higher-quality data, simulated robots demonstrated faster and more efficient learning, improving their overall reliability and performance.

When tested against other AI platforms, simulated robots using Northwestern’s new algorithm consistently outperformed state-of-the-art models. The new algorithm works so well, in fact, that robots learned new tasks and then successfully performed them within a single attempt — getting it right the first time. This starkly contrasts current AI models, which enable slower learning through trial and error. 

The research will be published on Thursday (May 2) in the journal Nature Machine Intelligence.

“Other AI frameworks can be somewhat unreliable,” said Northwestern’s Thomas Berrueta, who led the study. “Sometimes they will totally nail a task, but, other times, they will fail completely. With our framework, as long as the robot is capable of solving the task at all, every time you turn on your robot you can expect it to do exactly what it’s been asked to do. This makes it easier to interpret robot successes and failures, which is crucial in a world increasingly dependent on AI.”

Berrueta is a Presidential Fellow at Northwestern and a Ph.D. candidate in mechanical engineering at the McCormick School of Engineering. Robotics expert Todd Murphey, a professor of mechanical engineering at McCormick and Berrueta’s adviser, is the paper’s senior author. Berrueta and Murphey co-authored the paper with Allison Pinosky, also a Ph.D. candidate in Murphey’s lab.

The disembodied disconnect

To train machine-learning algorithms, researchers and developers use large quantities of big data, which humans carefully filter and curate. AI learns from this training data, using trial and error until it reaches optimal results. While this process works well for disembodied systems, like ChatGPT and Google Gemini (formerly Bard), it does not work for embodied AI systems like robots. Robots, instead, collect data by themselves — without the luxury of human curators.

“Traditional algorithms are not compatible with robotics in two distinct ways,” Murphey said. “First, disembodied systems can take advantage of a world where physical laws do not apply. Second, individual failures have no consequences. For computer science applications, the only thing that matters is that it succeeds most of the time. In robotics, one failure could be catastrophic.”

To solve this disconnect, Berrueta, Murphey and Pinosky aimed to develop a novel algorithm that ensures robots will collect high-quality data on-the-go. At its core, MaxDiff RL commands robots to move more randomly in order to collect thorough, diverse data about their environments. By learning through self-curated random experiences, robots acquire necessary skills to accomplish useful tasks.

Getting it right the first time

To test the new algorithm, the researchers compared it against current, state-of-the-art models. Using computer simulations, the researchers asked simulated robots to perform a series of standard tasks. Across the board, robots using MaxDiff RL learned faster than the other models. They also correctly performed tasks much more consistently and reliably than others. 

Perhaps even more impressive: Robots using the MaxDiff RL method often succeeded at correctly performing a task in a single attempt. And that’s even when they started with no knowledge.

“Our robots were faster and more agile — capable of effectively generalizing what they learned and applying it to new situations,” Berrueta said. “For real-world applications where robots can’t afford endless time for trial and error, this is a huge benefit.”

Because MaxDiff RL is a general algorithm, it can be used for a variety of applications. The researchers hope it addresses foundational issues holding back the field, ultimately paving the way for reliable decision-making in smart robotics.

“This doesn’t have to be used only for robotic vehicles that move around,” Pinosky said. “It also could be used for stationary robots — such as a robotic arm in a kitchen that learns how to load the dishwasher. As tasks and physical environments become more complicated, the role of embodiment becomes even more crucial to consider during the learning process. This is an important step toward real systems that do more complicated, more interesting tasks.”

The study, “Maximum diffusion reinforcement learning,” was supported by the U.S. Army Research Office (grant number W911NF-19-1-0233) and the U.S. Office of Naval Research (grant number N00014-21-1-2706).


Future direction: NoodleBot [VIDEO] | 

The published study includes tests performed with simulated robots. Next, they will test the algorithm on robots in the real world. They developed this snake-like robot, called "NoodleBot," for future testing.


Simulated robots learn in one [VIDEO] |

This video illustrates the single-shot learning capabilities of MaxDiff RL.


Although the current study tested the AI algorithm only on simulated robots, the researchers have developed NoodleBot for future testing of the algorithm in the real world.

CREDIT

Northwestern University