Deployable Multimodal Machine Intelligence: Applications in Biomedical Engineering

Contents
1 Preface and A Brief Guide to the Chapters
1.1 Steer DMs with Various Actuation Modalities
1.2 Tethered and Insertable DM/DS
1.3 Inflatable DMs: From Tethered to Untethered
1.4 Swallowable Magnetic DMs for Untethered Motions
1.4.1 Permanent Magnet Actuation for External Field Generation
1.4.2 Electromagnetic Actuation for External Field Generation
1.4.3 Untethered Magnetoelastomer
1.5 Wearable DMs
1.6 Deployable Sensing Mechanisms
1.7 Intelligent DMs with Multimodal Sensing
1.8 Future Perspectives
2 Orimimetic Folds into Deployable Mechanisms with Potential Functionalities in Biomedical Robotics
2.1 Introduction
2.2 Orimimetic Design and Its Role in Keyhole Procedures
2.2.1 Origami for Rapid Design
2.2.2 Action Origami and Its Role in Keyhole Procedures
2.3 Origami-Inspired Technologies
2.3.1 Miura-Ori-Inspired Designs
2.3.2 Curved-Crease Origami
2.3.3 Waterbomb-Inspired Designs
2.3.4 Modified Mountain/Valley-Fold Origami
2.4 Other Miscellaneous Origami Methods
2.4.1 Variably Patterned Graphene Structures
2.4.2 Variably Patterned Cell-Based Designs
2.5 Other Graspers
2.5.1 Two-Jaw Surgical Graspers
2.5.2 Issues with the Traditional Two-Jaw Graspers
2.6 Fortune-Teller-Inspired Grasper Designs
2.6.1 Modified Fortune Teller Design
2.6.2 Actuation Methods
2.6.3 Grasping Capability of Three Actuation Methods
2.6.4 Range of Motion and Grasp Coverage
2.6.5 Degrees of Freedom
2.6.6 Assembly from a Flat Surface and Flat Foldability
2.7 Remarks
References
Part I Tethered Insertable DMs
3 Deployable and Interchangeable Telescoping Tubes
3.1 Introduction
3.2 Related Work
3.2.1 Deployable and Collapsible Designs
3.2.2 Actuations for Folding Structures
3.2.3 Bistable and Locking Methods
3.3 Methods and Design
3.3.1 Bistable FITT Structure
3.3.2 SCAT with a Tongue Depressor and Tendon-Driven Swab
3.3.3 Tendon-Driven Mechanism (TDM)
3.3.4 Modularity of Design: Interchangeable Tips
3.4 Simulation
3.5 Force Analysis Experiments
3.5.1 Bistability
3.5.2 TDM Structure
3.6 Discussion
3.7 Conclusion and Future Work
References
4 Deployable Parallelogram Mechanism for Generating Remote Centre of Motion Towards Ocular Procedures
4.1 Introduction
4.2 Ophthalmic Surgery
4.3 Remote Centre of Motion
4.4 Comparison with Existing RCM Robot Mechanism
4.5 Kinematic Design Considerations
4.5.1 Design Goals (DG)
4.5.2 Design Preference (DP)
4.6 Proposed Design
4.7 Electrical Schematic Diagram
4.8 Experimentation Results and Observations
4.9 Weight of Main RCM
4.10 Belt and Pulley Backlash
4.11 Parts Assembly
4.12 Conclusion
4.13 Future Improvements
References
Part II Inflatable DMs: From Tethered to Untethered
5 Conceptual Origami Bending and Bistability for Transoral Mechanisms
5.1 Background
5.2 Prioritize the Needs
5.3 Design and Actuation
5.3.1 Overall Origami Deployable Structures
5.3.2 Origami Actuation Components & Bistability Rationale
5.4 Design Verifications
5.4.1 Material Tests
5.4.2 Usability Tests
5.4.3 Summary of the Overall System
5.5 Discussion
5.5.1 Needs-Metrics Table
5.5.2 Failure Mode Analysis
5.5.3 Risk Assessment Matrix
5.6 Conclusion
References
6 Tactile Sensitive Origami Trihexaflexagon Gripper Actuated by Foldable Pneumatic Bellows
6.1 Introduction
6.2 Design and Construction
6.2.1 Gripper Body
6.2.2 Actuation Mechanism and Construction Protocol
6.2.3 Working Principle of FlexagonBot
6.3 Sensor Working Principle and Calibration
6.3.1 Sensor Design
6.3.2 Sensor Working Principle
6.3.3 Sensor Calibration
6.4 Flexagonbot Payload Test
6.5 Payload Test Results and Discussion
6.6 Conclusions and Future Works
References
7 Biomimetic Untethered Inflatable Origami
7.1 Introduction
7.2 Related Work
7.3 Materials and Methods
7.3.1 Prototype Design and Specifications
7.3.2 Origami Exoskeleton Design
7.3.3 Valve and Arduino Setup
7.3.4 Reactant Compartment Design
7.3.5 Mechanism of SM
7.3.6 Paddle Fin Design
7.3.7 Proposed Tests
7.4 Results
7.4.1 Design Input 1—Inflation
7.4.2 Design Input 2—Heaving Motion
7.4.3 Design Input 3—Surge Motion
7.4.4 Design Input 4—Yaw Motion
7.5 Discussions
7.5.1 Feature 1: Inflation
7.5.2 Feature 2: Heave Motion
7.5.3 Features 3 and 4: Surge and Yaw Motion
7.5.4 Other Features
7.5.5 Future Applications
7.6 Conclusion
Appendix 1
Appendix 2
Full Arduino Code
Appendix 3
References
Part III Swallowable Magnetic DMs for Untethered Motions
8 Wormigami and Tippysaurus: Magnetically Actuated Origami Structures
8.1 Introduction
8.2 Wormigami Structure
8.2.1 IPM Magnet Placement
8.3 Wormigami Motion Capabilities
8.3.1 Caterpillar-Wave Motion
8.3.2 Rolling
8.3.3 Peristaltic
8.3.4 Downward Dog
8.3.5 Slinky
8.3.6 Hyperextension: “Head Lifting”
8.3.7 Inchworm Motion
8.3.8 Comparison of Movements of the Model
8.4 Tippysaurus Structure
8.5 Tippysaurus Motion Capability
8.6 Material Testing
8.7 Wormigami: Compression and Tensile Tests
8.7.1 Compression Test for Paper with Mod-Podge Without IPM
8.7.2 Compression Test for Paper with Mod-Podge Coating and IPM
8.7.3 Compression Ratio for the Plastic Model Without IPM
8.7.4 Tensile Test for Paper Model with Mod-Podge Without IPM
8.7.5 Tensile Test for Paper with Mod-Podge with IPM
8.7.6 Tensile Test for Plastic Without IPM
8.8 Tippysaurus: Compression and Tensile Tests
8.8.1 Compression Test for Paper with Mod-Podge Without IPM
8.8.2 Compression for Plastic Without IPM
8.8.3 Compression for Paper with Mod-Podge with IPM
8.8.4 Tensile Test for Paper with Mod-Podge Without IPM
8.8.5 Tensile Test for Plastic Without IPM
8.8.6 Tensile Test for Paper with Mod-Podge with IPM
8.9 Force Assessment
8.9.1 Contact Force on the Surface
8.9.2 Vertical Force Assessment
8.9.3 Overall Force Output
8.9.4 Unsupervised Contact Between External Magnet and Human Body
8.9.5 EPM Contact Monitoring
8.10 Conclusion and Remarks
References
9 Untethered Motion Generation and Characterization of Multi-Leg Insect-Size Soft Foldable Robots Under Magnetic Actuation
9.1 Introduction
9.2 Literature Review
9.3 Methodology
9.4 Results and Discussion
9.4.1 Wave Motion-Induced Along the Horizontal Plane
9.4.2 Compression of the Prototype
9.4.3 Lateral Extension with Respect to the Frontal Plane of the Prototype
9.4.4 Motion Along a Stable Board Surface
9.4.5 Motion Along an Irregular Surface
9.4.6 Flipping Over and Recovery of the Prototype
9.4.7 Future Directions of Study
9.5 Conclusions
References
10 Magnetically Actuated Luminal Origami
10.1 Introduction
10.2 Design of MALO
10.2.1 Robotic Origami Backbone
10.2.2 Magnetic Patterning and External Magnetic Field Generation
10.2.3 Motions Generated
10.3 Mechanical Tests
10.3.1 Tensile Test
10.3.2 Compression Test
10.3.3 Three-Point Flexural Test
10.3.4 Dynamic Force Analysis
10.4 Displacement and Speed Tracking
10.4.1 Omega
10.4.2 Peristaltic
10.4.3 Inchworm
10.5 Internal Deformation
10.5.1 Omega
10.5.2 Inchworm
10.5.3 Peristaltic
10.6 Surface and Environment Test
10.6.1 Waterproof Test
10.6.2 Surface Test (Gravel)
10.6.3 Surface Test (Gel)
10.6.4 Need-Metrics Matrix
10.6.5 Risk Assessment
10.7 Discussion on Potential Applications
References
11 Compressable and Steerable Slinky Motions
11.1 Introduction
11.2 Design Rationale
11.2.1 Design Progress & Overall Design
11.2.2 Square Slinkey
11.2.3 Deciding the Number of Folds
11.2.4 Materials Used
11.3 Motion Analysis
11.3.1 Inchworm Motion
11.3.2 Peristaltic
11.3.3 Rolling Motion
11.3.4 Head Rotation
11.3.5 Leaping Motion
11.3.6 Slinky Motion
11.3.7 Summary of Motion Capabilities
11.3.8 Reconfigurability Advantages
11.3.9 Mechanical Testing
11.4 Improvements and Potential Applications
11.4.1 Possible Improvements
11.4.2 Possible Uses
11.4.3 Other Design Possibilities
11.4.4 Computer-Aided Design (CAD)
11.5 Safety, Risk & Ethics Issues
11.5.1 Robot Overview
11.5.2 Risk Identification
11.5.3 Risk Management
11.6 Patent Review & Comparisons
11.6.1 Patent Search & Approach
11.6.2 Related Patents
11.6.3 The Design Novelty
11.6.4 Motion Comparison
11.6.5 Tabulated Needs and Metrics
11.6.6 Metric Comparison
11.7 Remarks
References
12 Magnetically Actuated Origami Structures for Untethered Optical Steering in Remote Set-up: Preliminary Designs and Characterisations
12.1 Introduction
12.2 Background
12.3 Design Considerations and Materials
12.3.1 Fabrication
12.3.2 Magnetic Actuation Characterisation
12.3.3 Optic Steering System Setup
12.4 Origami Designs
12.4.1 Starshade Origami Pattern and Structure
12.4.2 Nejiri-Ori Origami Pattern and Structure
12.4.3 Oricep Origami Pattern and Structure
12.4.4 Sarrus Origami Pattern and Structure
12.4.5 Twisted Tower Origami Pattern and Structure
12.5 Steering Methods
12.5.1 Magnetic Actuation of Origami Structures
12.5.2 Remote Magnetic Actuation of Nejiri-Ori Structure with PM and EM
12.5.3 Displacement Characterisation
12.6 Characterisation Results
12.6.1 Force Characterisation of Starshade Using the Force Sensor
12.6.2 Load Bearing Capability and Stiffness of Starshade Design
12.6.3 Starshade Reversibility Characterisation
12.6.4 Nejiri-Ori Reversibility Characterisation
12.7 Optical Component Steering
12.7.1 Direct Steering of Light Projection
12.7.2 Setups Indirect Beam Steering with Optical Reflective Surface
12.7.3 Indirect Steering (with Permanent Magnet) of Laser Beam Pathway
12.7.4 Indirect Steering with Electromagnet Nejiri-Ori Structure
12.7.5 Steering Other Origami Designs
12.8 Discussion
12.8.1 Manual and Magnetic Actuation
12.8.2 Electromagnet and Permanent Magnet
12.8.3 Optical Beam Steering Demo
12.9 Conclusion and Remarks
Appendices: Background Survey on Optical Component Steering Devices
References
13 Untethered Soft Ferromagnetic Quad-Jaws Cootie Catcher with Selectively Coupled Degrees of Freedom
13.1 Introduction
13.2 Methods and Materials
13.2.1 Model Inspiration
13.2.2 Materials Used
13.2.3 Model Design
13.2.4 Fabrication Method
13.2.5 Model Mechanism of Action
13.3 Methods and Results
13.3.1 FEA Simulations of Walking and Grasping Motion
13.3.2 Measuring Jaw Motion with Changing Magnetic Field
13.3.3 Measuring Grip Force Generated with Changing Magnetic Field
13.3.4 Walking Motion Analysis
13.3.5 Proof-Of-Concept Demonstration of the Anastomosis
13.4 Discussions
13.4.1 Advantages with the Untethered and Coupled DOFs
13.4.2 Limitations of Prototype
13.4.3 Other Envisioned Applications of the Proposed Model
13.5 Conclusion
References
Part IV Wearable DMs
14 Wearable Origami Rendering Mechanism Towards Haptic Illusion
14.1 Introduction
14.2 Related Work
14.2.1 Haptics in Virtual Reality
14.2.2 Pressure-Aided Transdermal Drug Delivery
14.2.3 Haptic Feedback and Materials
14.2.4 Concept of Magnetically Actuated WORM
14.3 Methodology
14.3.1 WORM Structure
14.3.2 Magnetic Actuation
14.3.3 Innovations
14.3.4 Movement
14.4 Results
14.4.1 Dynamic Force Analysis
14.4.2 Rotational Axis of the EM
14.4.3 Orientation of IMs
14.4.4 Location of EM
14.4.5 Location of Internal Magnets (IMs)
14.4.6 Vibration of WORM
14.5 Discussion
14.5.1 Significance of Results
14.5.2 Limitations
14.5.3 Future Improvements
14.5.4 Future Potential Applications
14.6 Conclusion
References
15 Deployable Compression Generating and Sensing for Wearable Compression-Aware Force Rendering
15.1 Introduction
15.2 Background
15.2.1 Anatomy of the Skin
15.2.2 Penetration Pathways for Drug Absorption
15.2.3 Transdermal Drug Delivery Technology
15.2.4 Wearable Haptic Systems
15.2.5 Origami Mechanism
15.2.6 Sensing Mechanism
15.3 Design Methodology
15.3.1 Origami Structural Design
15.3.2 Pressure Sensor Design
15.3.3 System Fabrication
15.3.4 Working Principle
15.4 Experiments
15.4.1 Pneumatic Origami Structure Motion Generation
15.4.2 Mechanical Test for Microfiber Sensor
15.4.3 Onboard Data Acquisition
15.4.4 Evaluation
15.5 Discussion
15.5.1 Improvements
15.5.2 Future Potential Applications
15.6 Conclusion
References
Part V Deployable Sensing Mechanisms
16 Kinesthesia Sensorization of Foldable Designs Using Soft Sensors
16.1 Introduction
16.2 Methods
16.3 Fabrication of the Soft Hydrogel Silver Nanowire Sensor
16.4 Results and Discussion
16.5 Conclusions
References
17 Flat Foldable Kirigami for Chipless Wireless Sensing
17.1 Introduction
17.2 Theory
17.3 Materials and Methods
17.4 Tag Antenna Characterization
17.5 Wireless Sensors
17.6 Discussion
17.7 Conclusion
Appendix 17.1: Literature Review
Appendix 17.2: Sample of .s1p File with Interpretation
Appendix 17.3: VNA Calibration Procedure
References
18 Deployable Kirigami for Intra-Abdominal Monitoring
18.1 Introduction
18.1.1 Needs and Significance
18.1.2 Current Routes of Measuring IAP
18.1.3 Patent Space
18.1.4 Related Sensing Technologies
18.2 Methods
18.2.1 Test Kirigami Geometry
18.2.2 Parameter Characterization to Optimize the Selected Geometry
18.3 Results
18.3.1 RRC Test to Find Out the Geometry with the Best RRC
18.3.2 Parameter Characterization to Optimize the Selected Geometry
18.4 Discussion
18.5 Conclusion
References
19 Stretchable Strain Sensors by Kirigami Deployable on Balloons with Temporary Tattoo Paper
19.1 Introduction
19.2 Related Work
19.2.1 Electronic Catheter Balloons
19.2.2 Kirigami Technique in Flexible Electronics
19.2.3 Intrinsically Flexible Conductive Materials
19.3 Materials and Methods
19.3.1 Phase I (Kirigami Design Cuts)
19.3.2 Analyze Kirigami Design Cuts of Phase I
19.3.3 Phase II (Adhere Gold Substrate to Balloon)
19.3.4 Finalize Construction Method
19.3.5 Analyze Both Fabrication Methods
19.4 Results and Discussion
19.4.1 Measurement of Normalized Resistance (ΔRR0) Against x-longitudinal Strain and y-axial Strain
19.4.2 Measurement of Pressure Against Volume
19.4.3 Setup Measurement of Air Volume Against Balloon Radius
19.4.4 Setup to Measure Resistance Against Volume
19.4.5 Normalized Resistance (ΔRR0) Against Radius Strain of the Balloon
19.4.6 Normalized Resistance (ΔRR0) Against Pressure
19.4.7 Normalized Resistance (ΔRR0) Against Volume
19.5 Conclusion and Future Work
References
Part VI Intelligent DMs with Multimodal Sensing
20 Multi-DOF Proprioceptive Origami Structures with Fiducial Markers
20.1 Introduction
20.2 Fiducial Tags in ML Estimations Using ArUco Markers
20.3 Crease Patterns
20.3.1 Pattern 1—3L1J (3 Legs 1 Joint)
20.3.2 Pattern 2—3L2Ja (3 Legs 2 Joints (a))
20.3.3 Pattern 3—3L2Jb (3 Legs 2 Joints (b))
20.3.4 Pattern 4—4L2J (4 Legs 2 Joints)
20.4 Calibrations
20.4.1 Triaxial Stiffness
20.4.2 Motion Estimation
20.4.3 Triaxial Load Sensitivity
20.4.4 Validation Using ATI-Nano
20.5 Results and Analysis
20.5.1 Motion Estimation
20.5.2 Force Sensitivity
20.5.3 Validation with ATI-Nano
20.5.4 Noise
20.5.5 Overview of Results
20.6 Discussion
20.7 Conclusion
Appendix 1: Python Codes
Appendix 2: Motion Estimation
Appendix 3: Force Sensitivity Graphs
Appendix 4: ML Sensor-ATI Overlapped Graphs
Appendix 5: Noise Graphs
References
21 Unsupervised Intelligent Pose Estimation of Origami-Inspired Deployable Robots
21.1 Introduction
21.1.1 Various Origami Motions
21.1.2 2D Feature Tracking
21.2 Visual Feature-Based Planar Motion Tracking
21.2.1 Vision-Based Trackers
21.2.2 Track Inchworm, Omega and Tumbling Motions
21.2.3 Future Alternative Spatial 6-DOF Tracking Using Aruco Markers
21.3 Sim2Real 6-DOF Pose Estimation Using Synthetic Data
21.3.1 Image Generation for Deep Learning-Based 3D Tracking
21.3.2 Domain Randomization
21.3.3 CDAE Network Architecture
21.4 Remarks and Alternative Approaches
References