Collaborative Task Scheduler for Robots

Shelf replenishment looks simple until product availability, customer flow, workforce capacity and store policies compete for attention at the same moment. This initiative explores how a multinational manufacturer can combine employees, mobile robots and store data to support more responsive convenience-store operations, beginning with the restocking and arrangement of PET bottles.
A convenience-store employee and a mobile robot with an articulated arm restock bottled drinks together in a clearly recognizable retail aisle.
Store-state visibility
Products, customers and employees sensed
Context-based scheduling
Tasks prioritized from current conditions
Predeployment simulation
Robot actions tested in a digital twin
Shared task execution
Employees and robots coordinate replenishment
A customer selects a bottled drink from a stocked convenience-store shelf while an employee and a mobile robot replenish products in the background.
Corroborative task schedular for robot and employees in convenience store

A corroborative task scheduler connects current store conditions with task allocation, helping employees and robots determine what requires attention next. The approach brings sensing, mathematical optimization, human-robot communication and digital-twin simulation into one operational concept.

Convenience stores are dynamic by nature. Products move, customers arrive, priorities change and shelf tasks compete for limited human and robotic capacity. The challenge is not simply to automate a task, but to decide which task matters now and coordinate its safe execution.
Objectives
The initiative set out to support store employees with multi-task robots equipped with arms and carts, using the restocking and arrangement of PET bottles as the initial target use case. Its central objective was to create a scheduling layer that could interpret the store's current situation, balance operational priorities and direct replenishment work to employees and robots. The source also identifies safety and efficiency before deployment as core requirements, supported through a digital twin of the store.
Opportunity
The concept creates a path from isolated robotic actions to a more adaptive operating model in which management decisions and frontline work are connected through data. Beyond the initial PET-bottle use case, the source points to broader exploration of multi-task robots and a Vision-Language-Action model. With the operating status and performance evidence still to be validated, the opportunity is to test whether the same combination of situational sensing, scheduling and simulation can support a wider set of store tasks while keeping people central to execution and oversight.
Turning a changing store environment into coordinated action

The initial use case required more than a robot capable of moving products. The initiative needed a reliable view of current store conditions, a way to prioritize competing tasks, a controlled method for testing robot behavior and a practical coordination model for employees and robots.

Key Challenges
High-angle view of a busy convenience store with customers, employees, shelves at different stock levels and a mobile robot moving through the retail floor.
1: Dynamic store visibility
The scheduling process needed to understand how many products were on the shelves and how many customers and employees were present. Without this situational context, task decisions could not reflect what was happening in the store.
A store employee moves a loaded replenishment cart from the stockroom as a mobile robot waits beside another trolley near the sales floor.
2: Situation-based prioritization
Convenience-store tasks are interdependent and can change with demand. The initiative required a way to prioritize restocking work against sales considerations, cost and store policies, an area that was outside the client's original team scope.
A robotics engineer observes a mobile manipulator being tested beside stocked retail shelves within a marked safety area in a store simulation environment.
3: Safe predeployment validation
Robot actions needed to be examined before they entered a live retail environment. The approach therefore had to support simulation of movement and task execution with safety and efficiency in view.
A convenience-store employee gestures toward a drinks shelf beside a mobile robot carrying a crate, with shoppers visible farther along the aisle.
4: Human-robot coordination
The operating model needed to preserve communication between employees and robots, allowing people to signal work, request help and remain part of task execution rather than treating automation as a separate workflow.
Solution
A context-aware scheduling layer for collaborative retail operations

The described solution combines sensing technologies, mathematical optimization and a digital twin to convert store conditions into task requests for employees and robots. Python and ROS are identified as supporting technologies, while the simulation approach draws on experience from manufacturing and robotics research and development.

check icon
Store-state sensing

Sensing technologies establish the operational context by capturing shelf product levels and the number of customers and employees in the store. This gives the scheduler a grounded view of current conditions.

check icon
Context-aware task scheduling

Scheduling technologies use the current situation to prioritize work while considering sales, cost and store policies. The system can then request restocking activity from employees and robots.

check icon
Digital-twin simulation

A digital twin of the store is used to simulate robot actions before deployment. The source positions this step as a way to examine safety and efficiency, referencing simulation practices already used in manufacturing and robotics research and development.

check icon
Collaborative task interaction

The operating concept supports task requests and communication between employees and robots. Employees can indicate what needs to be done, while the robot can collect shelf-status information and support assigned tasks.

Impact
A foundation for more responsive store operations

By connecting store management and frontline work through data, the approach can support better shelf availability, lower inventory pressure, reduced employee workload, safer execution and easier customer access to products. The source presents increased sales and decreased inventory as expected value, but provides no baseline, measurement period or quantified outcome. These benefits should therefore be treated as intended impact until validated.

Featured
success stories
Empowering field operations through digitalized service management and improving safety conditions trough connected worker technologies
View all success stories
Profit per square meter, now on the sales floor of 13 department stores in Mexico
Every square meter in a department store has a cost, and the sales team decides what fills each one. A Mexican department store retailer built a report with NTT DATA linking profitability to floor space across 13 stores and 11 divisions. Sales teams use the report to decide on space and follow each action on the floor.
Profit per square meter, now on the sales floor of 13 department stores in Mexico
Four business units, one technology roadmap after the merger
After an acquisition, a retail group ran four business units on four technology stacks. NTT DATA mapped processes, applications, ERP, CRM and infrastructure across all of them and designed a low-risk transition roadmap. The group now works from a prioritized portfolio of initiatives, with synergies identified for each unit.
Four business units, one technology roadmap after the merger
Drag