Remote Controlled Mining Interface
The aim of this project was to gain experience of the process of designing a complex product or system where both cognitive and physiological aspects are involved focusing primarily on professional users having expert knowledge and experience using the product.
Taking the mining industry, the key challenge was to design a remote controlled scenario, moving miners and the rock breaking task out of the mine to primarily ensure safety and improve work efficiency.
It was critical to experience the scenario firsthand and interact with professional users who have expert knowledge and experience using the equipment. We also drew parallels from other industries that had taken steps towards remote controlled operations such as medicine and aeronautical engineering. Working prototypes allowed us to iterate and develop our concept.
Our concept tries to bridge the gap between manual production and complete automation. It augments current remote operator technology and is safer, reducing machine wear and tear. Multiple rock breakers can be operated improving efficiency. This being a slight change from current operations, is a stepping stone to full automation.
Role in Team
I was an equal part of the research and idea generation phase, documenting user interviews and analyzing the information collected. I played an integral part in prototyping for the project. I worked along side our developer to make the models while he dealt with the coding bit. Also having had previous experience in creating wire frames and app design, I was a part of drawing out the wireframes for the touch screen interface. I contributed to scripting, filming and editing the videos and scenarios.
We have been mining for centuries and over years we have even gotten better at it and even efficient with the development of machines and computerized systems. It reduces manual labour, time , effort and making the job more precise. It has now come down to maximizing output and minimizing downtime, as without profits mining would not be a viable industry.
As we continue to dig deeper for minerals, there are certain challenges that begin to emerge: reserves are depleting and quality of the ore is declining. Also labour and energy costs are continuously rising. As we go deeper it takes twice as long to reach the bottom and back up to the surface. Is it commercially productive to continue to go down that road?
To address such a situation we have already begun by automating certain simple tasks to ensure consistency but there are still large number of activities that require human operation and supervision. Also based on subject matter experts, full automation is projected to the year 2020 and beyond. How can we in the meantime fill in the gap between operator controlled machines to autonomous systems?
Umeå Institute of Design
MFA Interaction Design
Duration / Year
10 Weeks / Fall 2015
Shigeo Katsura - Gordon
In Collaboration with
Swedish Interactive Institute
*Photos from this project are under legal review for release by the sponsor company, they will be made available soon.
Our concept tries to bridge this gap between manual production and complete automation, that can be implemented now. The concept augments current remote operator technology and is safer, reducing machine wear and tear. Multiple rock breakers can be operated improving efficiency. This being a slight change from current operations, is a stepping stone to full automation.
The solution involves a touch screen interface with a representational grid displaying rocks and the rock breaking arm. The position of rocks on the grizzly would be mapped onto the grid using camera vision. By touching the rock, the rock breaking arm moves into position, leaving the operator to use the joysticks to make minor precise movements to break the rock. This reduces a huge amount of time as the machine takes care of the general movement of the arm from one position to another and insuring reduced wear and tear and human error. It also reduces stress levels from having to operate the machine continuously.
The concept suggests two operational modes:
Manual operation involves an additional step of having to tap on each rock to move the arm into position and then using conventional joysticks to break the rock.
Semi Automatic Operation
In semi - autonomous mode the machine moves from one rock to another autonomously, deciding the best route for the arm, and the operator only controls the joysticks to break the rocks. This increases the efficiency and reduces time for the required process.
Future Rock Breaker Operator Scenarios
Using our concept we predicted the role of the future operators might change allowing them to perform multiple tasks. We outlined two possible operator roles, one of a full time rock breaker operator and the other of a supervisor, operating the machines occasionally.
Full time Operator
The video highlights the role of a full time operator, using multiple rock breakers in different mining locations. The interface shows the status and activity of the different sites allowing the operator to switch through them.
A supervisor operates the rock breakers as a secondary task. Rock breaking itself is a secondary task and only needs to be performed when the explosives have not efficiently broken the rock. The tablet interface allows the supervisor to perform his regular task and occasionally pitch in to break rocks stuck on the grizzly.
Initial Research and Contextual Inquiry
We conducted our initial research reading through articles and books to understand both the mining process, role of rock breaking and also the advantages and disadvantages of remote controlled operations. We established early guiding principles of efficiency, accuracy, reliability, safety and profits to help guide our direction and future concepts. We visited the Aitik mine for our contextual inquiry and found that operators in the mine had the same problems as remote operator of viewing the rock breaker pit through a camera view, unable to judge the depth through a flat view. The operators also expressed safety concerns while operating the breaker in close proximity to people or trucks. They were also expressed the need to operate the machines carefully, avoiding breakdowns as those would affect the targeted production goal.
As part of the project we visited Oryx Simulators in Umeå, who develop hardware and testing simulators for heavy machinery industry, to experience and control a rock breaker first hand. We soon realised how difficult it is perceive perspective and depth. Although there were three different views available, it was still extremely hard to maneuver the rock breaker. We also realised that since machines are being automated, operators can control multiple machines simultaneously, leading us to further develop our concept.
Task Analysis and Wireframing
Based on our research and finding we did a task analysis as a way to set which actions will be controlled by the touch screen interface. This helped discuss the priority of information and what will be displayed.
The prototype was made out of laser cut MDF sheets. The rock breaking arms were fitted with servo motors, connected to an arduino and using cameras to detect the rock. Light sensors were fitted into each grid position to detect when a rock placed on the grizzly. This is how we were able to automate the arm, moving it into position.
Testing with Stakeholders
Since we had a working model, we were able to explain our concept better and give our stakeholder to experience it first hand. This is was great as a prototype allowed a start to a discussion and future possible opportunities as opposed to seeing a simulation on video.
This being a professional project scenario where access to miners was limited and we had to find alternative ways to find information and identify problem areas. Also the for the scope of our project we may have delved deep into creating a working prototype so it can actually be validated, we did not put much focus on the final look and feel of the interface and considered the low fidelity wire frames as our final outcome.