As the dust settles on the latest AWS re:Invent conference, the ProCogia team has been reflecting on the latest announcements from the event. Perhaps the most exciting development of all is AWS’s commitment to Bioinformatics with Amazon Omics and the forecast of a ‘zero-ETL’ future, starting with the availability of direct integration between Amazon Aurora and RedShift. While these new offerings are not as mature or rich in features and integrations as more-established AWS services, they follow a growing trend of AWS pursuing the broadest offering of services for the lowest cost for the customer. On the other side of the spectrum, Amazon is capitalizing on the past ten years of designing its own silicon chips, resulting in further optimizations of cost and performance in the latest round of computer announcements. This means that customers can stay on top of their cloud infrastructure with regular upgrades and can pay big dividends in both time and money. As an AWS Advanced Tier Consulting Partner, ProCogia can help you in your AWS journey. These were just a few of the big announcements at the AWS re:Invent 2022 conference. Read on for the key takeaways from the event, and find out more about our AWS solutions here.
Analytics
AWS Glue Data Quality cuts down the time required for data analysis and rule identification from days to hours by automatically measuring, monitoring, and managing data quality in data lakes and across data pipelines. Amazon Security Lake automatically centralizes your organization’s security data from cloud and on-premises sources into a purpose-built data lake stored in your account. This purpose-built customer-owned data lake service helps organizations aggregate, manage, and analyze log and event data to enable faster threat detection, investigation, and incident response. QuickSight (Amazon’s serverless business intelligence service) has five new capabilities that enable customers to forecast and ask questions. The automated data preparation also makes it faster for customers to ask questions about their data. The new paginated reporting feature makes it easy for customers to create and share business-critical operational reports using the Amazon QuickSight interface. QuickSight’s in-memory engine now supports 1 billion rows of data, making it easier and faster to analyze and visualize large datasets. Customers can now programmatically create, manage, and edit Amazon QuickSight dashboards and reports to accelerate migrations from legacy systems. Amazon introduced DataZone, a new data management service that helps customers catalogue, discover, share, and govern data across their organization. Amazon Athena now supports Apache Spark, which will enable customers to get started with interactive analytics using Apache Spark in less than a second, instead of waiting minutes as previously. Amazon’s Redshift integration for Apache Spark also makes it easier for customers to run Apache Spark applications on data from Amazon Redshift using AWS analytics and machine learning services. Discover ProCogia’s Business Intelligence & Analytics solutions here.
Database
Amazon Aurora’s zero-ETL integration with Amazon Redshift enables customers to analyze petabytes of transactional data in near real-time, eliminating the need for custom data pipelines.
Application Integration
Amazon EventBridge Pipes enables you to integrate supported AWS and self-managed services into your application as event producers and event consumers in a simple, reliable, consistent, and cost-effective way. Now, users can create point-to-point integrations between event producers and consumers. AWS Step Functions now include a distributed map state, enabling large-scale parallel workloads such as the on-demand processing of semi-structured data from Amazon S3 buckets. This provides a serverless solution for large-scale parallel data processing.
Developer Tools
AWS Application Composer helps developers simplify and accelerate architecting, configuring, and building serverless applications. Using Application Composer, you can drag, drop, and connect AWS services into an application architecture by using the browser-based visual canvas.
Compute
It was announced that new EC2 instance types are available to support data-intensive workloads with the highest EBS performance in EC2. They also can handle up to twice as many packets per second (PPS) as earlier instances. These upcoming EC2 instance types will enable increased levels of performance, lower costs and power efficiency.
- Hpc7g instances featuring new AWS Graviton3E chips will deliver the best price performance for HPC workloads on Amazon EC2.
- C7gn instances featuring the new AWS Nitro Cards with enhanced networking will offer the highest network bandwidth and packet rate performance across Amazon EC2 network-optimized instances.
- Inf2 instances powered by new AWS Inferentia2 chips will deliver the lowest latency at the lowest cost on Amazon EC2 for running the largest deep learning models at scale.
Lambda SnapStart for Java functions was announced, designed to produce a 10x increase in start-up time at no extra cost. Explore how you can optimize your data management with ProCogia’s data engineering services here.
New Industries
Amazon introduced Omics, a service that helps bioinformaticians, researchers, and scientists store, query, and analyze genomic, transcriptomic, and other omics data. It also generates insights from that data to improve health and advance scientific discoveries.
Machine Learning
Amazon revealed that SageMaker, its end-to-end machine learning service, has eight new capabilities, including:
- SageMaker Role Manager makes it easier for administrators to control access and define permissions for improved machine learning governance.
- SageMaker Model Cards make it easier to document and review model information throughout the machine learning lifecycle.
- SageMaker Model Dashboard provides a central interface to track models, monitor performance, and review historical behaviour.
- SageMaker Studio Notebooks with a data preparation capability that helps customers visually inspect and address data-quality issues in a few clicks. They also now enable data science teams to collaborate in real-time and allow customers to automatically convert notebook code into production-ready jobs.
SageMaker’s Automated model validation enables customers to test new models using real-time inference requests. The new support for geospatial data enables customers to more easily develop machine learning models for climate science, urban planning, disaster response, retail planning, precision agriculture, and more. Found this Re:Invent 2022 round-up helpful? Be sure to have a look at our full range of partner solutions services here to discover how ProCogia can help your organization to build highly scalable data solutions. [km-cta-block padding=20 label=”Get in touch” block-classes=”has-white-colour” image=”http://procogia-main-site-co-uk.stackstaging.com/wp-content/uploads/2021/09/data-science-1.jpg” background-position=”top right” background-size=”cover” ] To find out more about Data Solutions and see the benefits [km_button link=”http://procogia-main-site-co-uk.stackstaging.com/contact/” classes=”cta-2″]Contact us[/km_button] or [km_button link=”tel:+14252307396″ classes=”cta-2″]Call us on 1 425-230-7396[/km_button][/km-cta-block]
Further Reading
- Join the Preview – AWS Glue Data Quality
- Preview: Amazon Security Lake – A Purpose-Built Customer-Owned Data Lake Service
- Announcing Automated Data Preparation for Amazon QuickSight Q
- Create and Share Operational Reports at Scale with Amazon QuickSight Paginated Reports
- New Amazon QuickSight API Capabilities to Accelerate Your BI Transformation
- Amazon DataZone Overview
- Amazon Athena for Apache Spark
- Amazon Redshift Integration with Apache Spark
- AWS announces Amazon Aurora zero-ETL integration with Amazon Redshift
- Create Point-to-Point Integrations Between Event Producers and Consumers with Amazon EventBridge Pipes
- Step Functions Distributed Map – A Serverless Solution for Large-Scale Parallel Data Processing
- Introducing AWS Application Composer
- New General Purpose, Compute Optimized, and Memory-Optimized Amazon EC2 Instances with Higher Packet-Processing Performance
- New Amazon EC2 Instance Types In the Works – C7gn, R7iz, and Hpc7g
- Accelerate Your Lambda Functions with Lambda SnapStart
- Introducing Amazon Omics – A Purpose-Built Service to Store, Query, and Analyze Genomic and Biological Data at Scale
- Next Generation SageMaker Notebooks – Now with Built-in Data Preparation, Real-Time Collaboration, and Notebook Automation
- New for Amazon SageMaker – Perform Shadow Tests to Compare Inference Performance Between ML Model Variants
- Preview: Use Amazon SageMaker to Build, Train, and Deploy ML Models Using Geospatial Data
