This month's article looks into three digital tools that engineers, scientists and technologists can leverage whilst working on their projects. I'll be looking at application ideas and use cases, sharing thoughts on pros and cons, and giving a bit of insight into how you can start working with them. I use these tools on a day-to-day basis, at Cloudforest, on engineering projects and applications.
These three tools are Web Services, Cloud Storage, and the Python Programming language.
Let's take a closer look.
Tool 1: Web Services
The term Web Services encompasses the different ways electronic devices and websites communicate and share information with one another using the internet.
The first thing that comes to mind is often website data communication, but web services also include the Internet of Things - the connection and information exchange between internet-connected devices, sensors and storage.
According to a survey by McKinsey, 25% of worldwide businesses now use IoT technologies, and 43 billion IoT devices are projected to be in use by 2023, so this is becoming a ubiquitous, embedded technology.
Having devices connected to each other solves obvious technical problems:
'What's going on?' - Realtime data acquisition
'What has gone wrong?' - Live fault reporting
'What might be going wrong?' - Predictive maintenance
Less obviously, opportunities are created when web services are used to synthesise and aggregate data from disparate sources, such as website or publicly-available information with data acquired directly from products in-use.
For example, a team running a new vehicle field trial can collect ambient condition data by streaming data from a web-based weather service, using GPS data generated by the trial vehicles. Or predictive maintenance data from sensors can be correlated with image data streamed from connected cameras or a drone.
One of the principle barriers to using web services is the skillset required to set up the system, which often isn't available amongst people undertaking the project. However, it doesn't take much to learn some basic web development skills - Microsoft Learn, Google Skillshop and Amazon Web Services all offer guided-learning introductions to this subject.
Tool 2: Cloud Storage Technologies
Cloud technologies can be used to store data remotely in readily-accessible, secure and highly configurable forms. Storage can be configured to work with many different mediums, such as databases, text files, binary files or images.
The data generated by modern engineering projects is becoming larger and more varied, creating multiple storage, analysis and security considerations. Advantages are created by corralling systems under one central umbrella - this is one of the Cloud's key strengths.
Whilst legacy data storage systems are often run from on-premises hardware, cloud databases are provided by third parties (Amazon, Microsoft, Google and others) and are provisioned with an array of support services such as data hubs, systems for data processing, and gateways to Machine Learning systems.
Many devices can be connected directly into a Cloud-based hub, including modem-enabled data loggers, IoT sensors and hardware control units, and legacy or 'island' data stored in spreadsheets can be uploaded with a bit of coding. This makes for a smooth, cohesive work flow, independent of data source or data type.
The core advantage delivered through Cloud storage is in creating a common gateway and processing funnel for projects with bigger and more-complex datasets. As with web services, the skillset required to implement an effective solution can be a barrier, and storage costs can be expensive for really large data sets.
All the main cloud providers have training modules on their cloud products, which are usually simple enough for non-programmers to follow. See the links above.
Tool 3: Python
Python is a high-level, general purpose programming language first invented back in 1991. It was designed from the outset to be easy to work with, easy to read and extensible.
Because of this, it has become a powerful utilitarian language for supporting technical projects:
It can be installed easily on a variety of machines, from Windows and Linux desktops to micro controllers in data acquisition systems.
It can be extended with open-source modules dedicated to technical computing and data analysis, such as numpy, pandas and matplotlib. These modules can enable functionality equivalent to Matlab or other analysis packages.
It has numerous frameworks for building web services, configuring devices and gateways, and working with other software systems(eg. Excel).
It has become one of the de facto languages of machine learning - Many machine learning systems are now designed and executed in Python.
It is considered one of the easiest high-level programming languages to learn and start working with.
It's open-source - No licence fees, and a strong community available to develop the language and provide support.
These advantages have lead to python becoming one of the world's most widely-used languages: The Tiobe Index (for March 2020) has Python as the third most popular language in the world for programmers and projects, behind Java and C.
At Cloudforest, we use python regularly across a range of applications, including collecting information from the internet, processing and analysing acquired engineering data, and transforming data from one format to another (for storage, for example).
We also build engineering simulations or run calculations in Python, and we configure controllers (such as the famous Raspberry Pi) with Python libraries. You can work on nearly every layer of a technology development program in Python ...
To learn the language, the best resources include tutorials on the Python.org website, and numerous youtube videos. Once a little bit of skill has been acquired, the absolute best way to progress further is to start applying it to your own technical problems.
In Summary
So that's an introduction and overview of three tools that you can put to work in an engineering or product development 'technology stack' - Each tool can deliver advantages on a given project, and all three can work together to deliver quite dramatic improvements in project delivery (quality improvements, cost and lead time reductions). Hopefully this has been insightful, and delivered some food for thought.
This article marks a mini-pivot here at Cloudforest, where we have started consulting on engineering data analysis and modelling using digital skills we've developed building apps like Macadamia. We are big advocates of digital utilisation to accelerate engineering projects and programmes, from early-stage technology concepting and development through to product and process optimisation.
As always, thanks for reading and please feel free to share thoughts, ideas and questions below.
Comments