Additive Manufacturing (AM) technologies have shown a rapid development over the last decade. Metal AM is now adopted in numerous industries, such as naval, energy, aerospace and automotive. AM provides a very flexible solution for processing of exotic materials (Ni-alloys, Titanium, etc.) and enables repair of components that would not be possible with conventional processing. However, AM cannot provide on its own parts that will meet the requirements of the aforementioned industries. To this end, subtractive processes have to be combined in a hybrid solution, in order to achieve the required quality for the part, in terms of dimensional accuracy and surface finish. HybridR is proposing an innovative robotic-based platform, which incorporates Laser Metal Deposition, milling and grinding as enabling technologies. The use of a robot as a base structure will provide a very high flexibility to the system and will showcase that robots can have a very high potential in the manufacturing industry.

HybridR

A Hybrid Processing Robotic System

HybridR is proposing an innovative hybrid (additive and subtractive) robotic-based platform, which incorporates Laser Metal Deposition, milling and grinding as enabling technologies.

Keywords:
3D Scanning, Point Cloud Registration, robot, embedded machine vision, PLC, IoT devices, 3D reconstruction, ICP method, welding defects detection

Vision

Additive Manufacturing (AM) technologies have shown a rapid development over the last decade. Metal AM is now adopted in numerous industries, such as naval, energy, aerospace and automotive. AM provides a very flexible solution for processing of exotic materials (Ni-alloys, Titanium, etc.) and enables repair of components that would not be possible with conventional processing. However, AM cannot provide on its own parts that will meet the requirements of the aforementioned industries. To this end, subtractive processes have to be combined in a hybrid solution, in order to achieve the required quality for the part, in terms of dimensional accuracy and surface finish.

HybridR is proposing an innovative robotic-based platform, which incorporates Laser Metal Deposition, milling and grinding as enabling technologies. The use of a robot as a base structure will provide a very high flexibility to the system and will showcase that robots can have a very high potential in the manufacturing industry.

3D Scanning System with Pattern Projector

A 3D scanning system with a pattern projector is a non-contact method of acquiring the surface geometry of an object. The system involves projecting patterns onto the surface of the object and capturing the distorted patterns using cameras. These patterns are then used to reconstruct the 3D surface geometry of the object.

Components of the System A 3D scanning system with a pattern projector comprises of the following components:

Pattern projector

This is a device that projects a set of known patterns onto the surface of the object being scanned. The patterns are typically sinusoidal or binary-coded. The projector can be either structured light or digital light projection.

Cameras

Two cameras creating a stereo pair to capture images of the patterns projected on the object.

Ubuntu PC

This unit controls the projector and cameras, captures and stores images, and processes the data obtained to reconstruct the 3D surface geometry of the object.

Rototilt Table

The rototilt table is a motorized platform capable of rotating and tilting an object to different orientations. It provides controlled movement for the object during the scanning process, allowing multiple viewpoints to be captured.

Control Unit (Olimex)

The control unit manages the movement of the rototilt table and coordinates the scanning process. It communicates with the 3D scanners and stores the acquired point cloud data.

Point Cloud Registration Software

The system utilizes point cloud registration software to align and merge the individual point clouds captured from different viewpoints. This software employs algorithms such as Iterative Closest Point (ICP) or feature-based registration techniques to accurately register the point clouds.

vision_box_coordinate_system
3D Scanning System with Pattern Projector
Functionalities

The HybridR system is employing dedicated monitoring and control systems for each of the processes that are involved with the workflow, which increases its robustness and enable the delivery of a defect-free part at the end of the production.

Moreover, interfaces between the whole CAx chain are developed to enable the automatic toolpath generation based on the data that are acquired by the vision-based metrology system.

Material Deposit

Deposit material for green field manufacturing, feature addition on existing parts or repair.

Geometry Reconstruction

Utilize vision-based metrology for generation of part point cloud and advanced algorithms for automatic geometry reconstruction.

Parts Finishing

Utilize subtractive processing for finishing of the parts.

Goals and Objectives

1.

Create a complete set of data for different “smart city” and “smart surveillance” scenarios, which will be captured in real-world conditions from sensors in different locations and with different environmental conditions.

2.

Develop machine vision technology based on artificial intelligence and deep learning that will be trained with real-time data as well as with existing data sets.

3.

Exploit of deep learning techniques and IRIDA’s know-how in training of neural network modules and real-time implementation of deep learning architectures for commercial purposes, which have been gained through collaboration with a companies such as Qualcomm, Analog Devices, Arrow Electronics, Cadence and Xilinx.

4.

Pilot implementation of the functionalities in embedded systems (edge devices) using heterogeneous computing techniques to optimally exploit available computing resources such as CPU, GPU and DSP.

Methodology - Project Phases

Phase 1: Data Collection and Deep Learning Training

Dataset development and selection

Training of CNN / deep learning architectures

Phase 2: Embedded System Prototyping

Embedded platform selection

Embedded software development

Phase 3: Demo System Development and Validation

Testing based on datasets of Phase 1

Testing and validation in real-world conditions

Work Packages

Gantt SmartAEye

Impact

The SmartAEye product targets the “smart cities” and “smart surveillance” markets. The implementation of SmartAEye library, running in an integrated system (edge device), results in the creation of an extremely competitive product compatible that does not violate personal data (GDPR compliant). A typical example of the use of technology is the smart parking application: with the use of a “sensor” that integrates the functions of SmartAEye, the scenario can be implemented in which the citizens are informed about the empty parking spaces.

Result based on sensor and lens of the vision box

These functionalities are particularly critical for the implementation of “smart surveillance” applications where, for example, in shopping malls, shops and malls they measure the number and flow of customers without collecting personal data.

Methodology

SmartAEye Methodology
Stages of training a neural network

Model optimization and conversion process

  • Quantization
  • Pruning
  • Operation fusion
  • Memory optimization
  • Numerical representation

These are implemented through OpenVINO Toolkit or OpenVINO DL Workbench

SmartAEye Training

Results

Welding Defects Detection based on Point Cloud Alignment

We use density-based clustering to compare sets of data points and pinpoint irregularities to detect welding defects by utilizing 3D scanner data.

Input (Object Scan)
scan-object
Sampling
Sampling
Sampling
Pointcloud matching (ICP)
ICP
ICP
Distance filtering
Distance filtering

A 3D reconstruction of the object was accomplished by employing structured light 3D scanning. The result was a dense point cloud, which was subsequently transformed into a less dense point cloud.

The comparison was made between point clouds derived from both compliant and non-compliant CAD models, eliminating uncertainties typically found in actual components. The analysis aimed to determine the deviations associated with each error by comparing the point clouds and investigating their relationships.

Dissemination

During the period from 12/05/2020 to 11/05/2022, the project team dealt with the diffusion and commercial exploitation of SmartAEye through direct meetings with companies in Greece and Europe. In addition, the company participated in international exhibitions abroad, either as an exhibitor or as a guest of its international partners.

Participation in the DFAE meeting of Renesas in Dusseldorf

Irida Labs, in May 2022, as part of its collaboration with Renesas Electronics, to promote the results of the SmartAEye project, was invited to participate in the Distributors and FAEs meeting organized in Düsseldorf. This invitation continued the collaboration with Renesas as Irida Labs is a member of the preferred partner program.

Participation in the HikVision Innovation Summit exhibition

H Irida Labs has developed an essential collaboration regarding promoting the results of the SmartAEye project with HikVision, one of the world’s top 3 camera manufacturers. As part of this collaboration, in May 2022 HikVision invited selected partners, including Irida Labs, to present solutions, such as those developed within the SmartAEye project, at an exhibition organized in Amsterdam specifically for this purpose. HikVision promoted this exhibition with a particular website.

Contact

During the period from 12/05/2020 to 11/05/2022, the project team dealt with the diffusion and commercial exploitation of SmartAEye through direct meetings with companies in Greece and Europe. In addition, the company participated in international exhibitions abroad, either as an exhibitor or as a guest of its international partners.





    Funding Details

    This project has been co-financed by the European Regional Development Fund of the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH – CREATE – INNOVATE (project code: Τ2ΕΔΚ-03896).

    Skip to content