Home Group Projects Telepresence Publicity Funding Glossary Site Map Links wee_logo.gif (10353 bytes)


Related
Subjects...


CVCCam
EPSRC Summary
Recent Work
Systems
Past Work
Archive

Up


Mobile Hybrid Virtual Reality and Telepresence for Planning and Monitoring of Engineering Projects

Objectives and Background

The project had the following objectives. 1) To identify the scope for possible applications of hybrid virtual reality and telepresence coupled with mobile telecommunications, using the civil engineering industry as an exemplar for more general applications. 2) To explore existing analogous applications and to benefit from the experience of other industries. 3) To devise a methodology for use in construction, based on the new approach coupled with the application of video, voice, and data transfer to and from a remote vehicle and or fixed installation. 4) To illustrate the application of the developed methodology to practical situations in close co-operation with industrial partners through the creation and application of a prototype system. To achieve these objectives a system prototype will be designed, constructed, and implemented in a Civil Engineering application.

This two year EPSRC sponsored project integrated the work of three Departments within the University and utilised the expertise and resources of three progressive industrial organisations. Although the physical location for most of the work was the Transparent Telepresence Laboratory, the Departments involved were; Design, Manufacture and Engineering Management (DMEM), Computer Science, and Civil Engineering. The collaborating companies were; Orange Personal Communications, Alvis Logistics, and Babtie Engineering. The resulting interdisciplinary project has produced a novel system combining virtual reality, telepresence, and mobile telephone technology, that has been very well received by industry and, via the media, the general public. The system has been demonstrated to the industrial partners on a number of occasions, to a more general industrial and academic audience at conferences and seminars, and to the general public via the media and public demonstrations. In November 1998, while participating in a SPIE conference, the system was demonstrated in operation between Boston Massachusetts and the University of Strathclyde in Glasgow. Using a small sub-notebook PC and mobile telephone in Boston, one of the system's motorised camera platforms was driven in Glasgow, this platform sent back live video to be displayed on the hand-held PC. The fact that all of this was done over the mobile telephone system made this transatlantic demonstration of very low bandwidth teleoperation and video a world first, and it clearly showed the potential of the system for the planning and monitoring of remote projects using mobile communications. In December 1998, at a workshop held at DERA in Chertsey, the bomb disposal vehicle located in the University in Glasgow was driven via the mobile phone system. Live video was transmitted over the mobile phone link from the camera platform on the vehicle and displayed on the laptop PC. The results of the project are unique. The work performed on stereo co-encoding represents a significant advance over the monocular encoding originally used in the system. We are sure that no other work on telepresence has used commercial mobile telephone technology as the communication medium, linked control of head slaved stereo camera platforms, mobile vehicles, and incorporated virtual reality, into one integrated system. The project has shown the viability of the hybrid system concept and has stimulated considerable interest among the industrial collaborators, other industrial companies, and the public in general. The illustration below shows the project concept, interested parties may obtain a copy of a video of the project by contacting g.m.mair@strath.ac.uk.

vr_poster.jpg (92670 bytes)

The Hybrid Mobile System A mobile, real-time, 3D-hybrid VR/TP system has been researched and formulated. Two prototypes of the system have been built and tested in real site environments. This integrated system incorporates emerging mobile telecommunications technologies to give rapid and easy access to real and virtual construction sites from arbitrary locations. It also allows remote surveillance of the construction site, and integration of real world images of the site with virtual reality representations, derived from planning models, for progress monitoring. The system consists of three main parts: 1) A Telepresence module for acquiring in real time real world data on the remote site. 2) A Communications module for linking a construction site with remote users. 3) A VR module for planning and scheduling of construction projects as well as a platform deployment.

The Telepresence module. In this project two camera platforms were developed building on earlier experience gained within the Transparent Telepresence Research Group (TTRG). The first is operated from a fixed position and has a simple pan and tilt monocular camera system with zoom capability. The compact camera, drive, and control system may be powered either from a mains or battery supply. The second platform is mounted on a remotely controlled tracked vehicle, which is provided by Alvis Logistics. This platform has a stereo camera pair capable of pan, tilt, and roll movements and is powered from the vehicle batteries. The vehicle and platforms at the remote site are controlled from a home site. The home site uses either desktop, laptop, or sub-notebook computers with the option of using a head mounted display with a head tracking device. The head tracker output can be used to slave the camera platforms to the user's head movements. The VR module. This module allows creation of a 'virtual construction project' from a schedule and subsequent visual monitoring of, and interaction with, the progress of the simulated project. It is based on the prototype developed at Strathclyde's Virtual Construction Simulation Research Group (VCSRG) and was further developed during the research. A fully operational VR model was devised of the mobile vehicle with all the required interfaces to manipulate it from the virtual world. This includes an interface for camera manipulation. One can manipulate the camera on site through manipulation of the 'virtual camera' on this interface. Site access construction data was supplied by Babtie Engineering. The Communications module. The user of the system is linked to the remote site via a communications channel. A wireless system has been selected using conventional mobile telephones supplied by Orange Personal Communications. In order to send video images over the mobile telephones, video compression technology is required. This is based on software developed in the Computer Science Department, University of Strathclyde, and optimised for the low bandwidths of mobile telephones. Both monoscopic and, through the developed stereo co-encoding, stereoscopic pictures can be transmitted for display on a VR Headset or user's screen. This research devised developed, tested and demonstrated a generic approach to remote control and monitoring of large engineering sites. This approach can be transferred to many other areas, such as surveillance, security, and work in any hazardous or difficult to access environment. Both the project progress monitoring and site control can be performed remotely and examined interactively providing more efficient use of, for example, the manager's time. Comparison using contiguous presentation of 'actual vs. planned' situations allows remote site inspection and progress verification by any party involved. A special programme/interface was developed to operate the VR camera so that one can choose the view in the virtual world and then connect to the real site - the camera will go to the position of the VR camera automatically. When the connection is made the real camera can be operated interactively through directing the VR camera. Two virtual worlds were developed. A model of the Telepresence laboratory is being used to demonstrate the communication module and remote control of camera. Connections to the lab have been made for a number of demonstrations, e.g. from Helsinki and Stanford using the Internet and from Boston and Chertsey using the mobile phone. Also a site model was developed as a day by day simulation of the construction of a real 8-story hotel located near to the University. This was used as a test-bed for site tests and presentations. The building and activities are modelled on one to one scale (there are more than 5000 objects) thus creating a unique database of 4D information for further research. The project has advanced the state of the art in telepresence system design through the incorporation of very low bandwidth communication links for control and video, and the creation of compact, low-cost, stand alone systems and the development of low bandwidth stereo co-encoding. Also the applications using the remote sensor platform on-site have proven their potential use to industry.

Publication and Dissemination of Results

The progress and results of this project have been disseminated widely. The following is a list of the dissemination activity directly attributable to the project.

Publications

Hybrid virtual reality and telepresence utilising mobile phone technology; G.Mair, J. Clark, R. Fryer, R. Hardiman, D. McGregor, A. Retik, N. Retik, K. Revie; Proceedings of SPIE, Telemanipulator and Telepresence Technologies V, Boston, Massachusetts, 4-5 November 1998. pp 60 - 70. ISSN 0277-786X, ISBN 0-8194-2985-6.

Integrated Telepresence, Virtual Reality, and Mobile Communications - A Project Report; G. Mair, J. Clark, R. Fryer, R. Hardiman, D. McGregor, A. Retik, N. Retik, K. Revie; Proceedings of Mechatronics '98, pp 805 - 810; Skovde, Sweden, 9-11 September 1998, ISBN 0-08-040009-1.

Integrating Virtual Reality and Telepresence to Remotely Monitor Construction Sites: a ViRTUE Project; Arkady Retik, Gordon Mair, Richard Fryer, Douglas McGregor. In "Artificial Intelligence in Structural Engineering" Ed. Ian Smith. pp 459 - 463 Springer-Verlag, Berlin, 1998, ISBN 3-540-64806-2.

Mobile Hybrid Virtual Reality and Telepresence for Planning and Monitoring of Engineering Projects ; A. Retik, J. Clark, R. Fryer, R. Hardiman, D. Mcgregor, G. Mair, N. Retik, K. Revie. UKVRSIG '97, Proceedings of the 4th UK Virtual Reality Special Interest Group Conference, pp 80 - 89, 1 November 1997, Brunel University. ISBN 1-872166-64-4.

The Transparent Telepresence Research Group, Gordon Mair. This is to be published in The Industrial Robot Journal, in May or June 1999, and is mostly comprised of a description of this project.

Teleoperation of an Unmanned Vehicle Using Mobile Phones. This is an article, written by G. Mair, to be published in the international magazine "Unmanned Vehicles", May 1999. Integrating Visual Scheduling and Telepresence for Monitoring of Construction Progress, (in preparation) Journal of Civil Engineering, May 1999.

Demonstrations and Media Reports

January 1999, Demonstration and presentation, by invitation, Center For Integrated Facility Engineering, Stanford University

18th December 1998 Demonstration to public during Geographical Society Christmas Lecture, University of Strathclyde, Glasgow, UK

2nd December1998 Long range demonstration at DERA, Chertsey, UK, to SILVER group.

24th and 25th November 1998 Long range demonstrations at TMA North, Edinburgh UK.

16th-17th November 1998 Long range demonstration at EPSRC MNA workshop Leeds University UK

4th November 1998 Transatlantic demonstration at SPIE conference, Boston Massachusetts, USA.

5th-7th October 1998, Long range demonstrations at TMA31, Brighton, UK.

24th June 1998 Public demonstration at Strathclyde University at Virtual Environment Laboratory launch.

1998 At various times practical demonstrations were provided at Babtie Engineering and Paterson Architects premises.

1998 "Integrating VR and Telepresence for effective Project Planning and Monitoring", invited seminar, VTT, Helsinki, Finland

1998 "Navigation and Positioning of Integrated TP/VR System", Parma University, Italy, trip funded by Royal Society of Engineering.

1998 "Planning and Designing construction Projects Using VR and Telepresence", UMIST.

20th May 1998 System and ongoing work demonstrated to visiting EPSRC representative.

18th-19th May 1998 System and ongoing work demonstrated to Shell, Lowestoft, UK.

April 1998 Presentation of project at Scottish Engineering Centre Glasgow UK

March 1998 Presentation of project to Scottish Optoelectronics Association, Glasgow UK

February 1998 Demonstration, by invitation, at Tomorrow's World Live Show held at the TWL Exhibition, Birmingham, approximate audience over the show period was approximately 10,000.

February 1998 Presentation of project at BMVA Symposium, London, UK

November 1997, Demonstration of the project work to date at EPSRC MNA workshop, University of Warwick, UK.

October 1997 Demonstration of the project work to date at IMAS meeting, Glasgow UK

April 1997 Earlier work of the Transparent Telepresence Group was demonstrated on Tomorrow's World television programme, this approximate 5 minute item had a UK and BBC World Service audience of around 150 million. Although the BBC paid for the costs, this demonstration of the principles of telepresence would not have been possible without the continuity afforded by the EPSRC grant.

Summary

Over the two year period of the project a large number of demonstrations have been provided for the media, industry and the public in general. The fixed pan and tilt system has been demonstrated on a live construction site together with the VR model, this has been repeated several times "on request" to industrial and architectural professionals, the world first transatlantic demonstration was a good example of this part of the system. Also the remote link from the HMD (head mounted display) to the remote head has been demonstrated on several different occasions. Finally a video of the project has been made to distribute to interested parties.

Newspaper and Magazine Reports

The Times, September 28th 1988, "Dials destined to change the world"

Impact, March 1998, "Mobile virtual reality is combined with telepresence"

The Scotsman, 10th February 1998, "New robot develops a mind of its own".

Real Time Graphics, Vol.6 No.5, Dec. 1997, "Remote VR head developed in Scotland". (This is a US based publication)

Engineering, Oct. 1997, Front cover illustration and work reported within article on robotics research in UK.

The Herald, 18th January 1997, "Get ahead in robotics".

The Sunday Times, 12th January 1997, "Robot head to beat bombs".

Further media coverage is expected as we further demonstrate our latest system in the near future.


indbul2d.gif (928 bytes)  See Also: CVCCam | Recent Work | Past Work | Future Work | Archive


Copyright 2004 Transparent Telepresence Research Group