Mar 06, 2020 newsletter

Rise of the machine learning engineer

Machine learning engineers

Kubeflow, a machine learning toolkit for Kubernetes, announced the launch of version 1.0 of its platform. Kubeflow’s first major release marks a massive shift toward optimizing and advancing the growing field of machine learning engineering.

What is it? Kubeflow is an all-in-one tool for full-stack machine learning engineering powered by Kubernetes.

With Kubeflow, engineers and data scientists can develop, build, train, and deploy machine learning models. Kubeflow simplifies and intertwines many of these workflows—turning data science from a mishmash of tools into a robust set of workflows that more closely resembles common development practices.

Developers can access a central dashboard to manage notebooks, docker images, and more through a user interface. Kubeflow directly integrates hosted Jupyter notebooks, a CLI for deployment and upgrades, and a profile controller for multi-user team management.

Why you should pay attention: Originally open sourced in late 2017, Kubeflow has expanded rapidly with contributions from more than 30 organizations, including industry heavyweights like Alibaba Cloud, Ant Financial, AWS, Google, and Microsoft.

Kubeflow is helping to power the rise of machine learning engineers. That means data science workflows will be more in line with existing developer workflows—using containers, CLIs, pipelines, YAML configuration files, and more.

What’s next: The Kubeflow team is planning to roll out new pipelines for defining complex machine learning workflows, additional metadata for tracking jobs and models, and more.


Who provides the best AI cloud services for developers?

Magic Quadarant

Gartner released a new Magic Quadrant—a ranking system for technologies—highlighting the best cloud providers for artificial intelligence developer services.

It's a growing opportunity: Gartner estimates that "by 2023, 40 percent of development teams will be using automated machine learning services to build models that add AI capabilities to their applications, up from less than 2 percent in 2019."

AWS leads by a small margin. Gartner praised AWS for its wide range of services—including Sagemaker and AutoPilot, which can automatically generate machine learning models. Gartner docked points from AWS for its sometimes confusing portfolio of tools. Research also showed that the cost of execution can vary between development and production environments, making it difficult for developers to model true costs.

Google falls behind AWS. Researchers commended Google’s strong language services and machine learning model inspection capabilities. The report, however, penalized Google for the lack of maturity in its cloud platform, which is still rapidly changing.

Microsoft competes with Google. Microsoft’s AI developer services were easily deployed to different environments, including Azure and on-premises. Microsoft, however, lacked a clear strategy, forcing developers to navigate across different brands and businesses—like Cortana and Azure—to discover what tools were needed.

The rest of the pack is thin: Other companies analyzed in the research report include H20.ai, Prevision.io, Salesforce, and Tencent. Gartner did not consider any of these platforms to be leaders.

For developers looking to tap into AI cloud services, the biggest tech companies—Amazon, Google, and Microsoft—hold a commanding lead over the rest of the ecosystem that is difficult to ignore—and is likely to continue growing.


Engineers are rethinking database development responsibilities

Database development

Redgate, a software provider of database technologies, released a new survey on the State of Database DevOps to better understand how engineering teams run their databases and what roles developers play. A few key takeaways:

Developers are leading database work: According to the survey, 55% of respondents said developers are solely responsible for authoring database changes. About 80% of respondents said developers are involved in authoring databases in some way—even if they’re not leading the changes.

That means more developers are taking on more roles outside of traditional development responsibilities: in 78% of companies, developers are responsible for both application and database development.

Automation is increasing: While developers are doing more database development, automation is replacing repetitive tasks. In 2018, just 9% of respondents were using automated database provisioning. That number has more than doubled to 21% over the past two years.

Deployment automation saw an even more dramatic jump. In 2018, 21% of respondents used some form of automated database deployments. That number has skyrocketed to 46% this year.

How teams can improve: The report also offers a few findings on what makes a more successful engineering team.

  • Add database version control: Deployers who use version control report lower production defect rates. About 33% of teams without version control said that more than 10% of deployments needed hotfixes, while just 24% of teams with version control said the same.
  • Make it easy to get code reviewed: Of teams that felt it was difficult to get database code reviewed, 37% said that more than 10% of deployments were defective. When teams made code reviews easy, just 19% said that more than 10% of deployments had defects.

As developers increasingly play a role in database changes, teams will need to rethink traditional workflows to better support cross-team collaboration.


Jamstack gets a boost with Netlify's rapid growth

Netlify Jamstack

Netlify, a cloud computing and hosting service for static websites, raised $53M in its recent Series C round. With its latest round of funding, Netlify plans to accelerate its battle against monolithic web architecture.

Nearly one million developers. Netlify boasts more than 800,000 developers on its platform, a huge leap from its 300,000 developers back in 2018. Consumers are gaining exposure, too: it's estimated that 8% of the internet population visits a Netlify-hosted site each month.

Netlify is redefining web development: Netlify lets developers quickly create websites by connecting Git repositories to its global application delivery network through continuous deployment.

Developers simply push code for their static websites to a source code repository. Netlify listens to changes in a codebase, triggers a new build, and then distributes that updated website to its servers.

Welcome to the Jamstack: Netlify has grown in popularity alongside the Jamstack—a modern modern web development architecture based on client-side JavaScript, reusable APIs, and prebuilt markup. As a result, developers don’t need to worry about infrastructure.

From the CEO of Netlify: "We're not trying to make managing infrastructure easy. We want to make it totally unnecessary."

Will it work? Static sites have made serious progress over the last few years, providing a viable alternative to server-side tooling like WordPress, Django, and Ruby on Rails.

Still, WordPress installations alone account for 37% of all websites, leaving plenty of room to grow for static websites. Netlify’s revolution is likely just beginning.


Small bytes

  • Sourcegraph, a universal code search tool, raised $23M in additional funding. As development stacks become more complicated with more tools, languages, and services, Sourcegraph hopes to let developers more easily navigate their codebases [VENTURE BEAT]
  • UnitQ, a new company that uses natural language processing to locate bugs mentioned by users on social media and in app reviews, raised $11M in funding. UnitQ integrates directly with Jira, simplifying the bug discovery and triaging process [VENTURE BEAT]
  • Unity announced a new student plan that will give developers free access to all the tools for professional AR/VR game production. In 2018, Hired—a recruiting platform—saw a 146% increase in demand for game development roles and a 1400% increase in demand for AR/VR roles [UNITY]
  • Microsoft updated Visual Studio Online—its newest remote development tool—with better Go, Python, and Docker container support. Launched last year, VS Online gives developers an easy way to access their code from anywhere, but enters a growing field of remote coding tools [ZDNET]
  • A recent report from accessiBe, revealed that 98% of the scanned web pages in the US failed to pass the Web Content Accessibility Guidelines compliance requirements. That means almost all websites it analyzed are exposed to potential lawsuits [ACCESSIBE]
  • A recent RedMonk programming language ranking that analyzes GitHub repositories discovered that Python is now the second most popular language. That marks the first time the second place spot has been held by a non-Java or JavaScript programming skills [REDMONK]
  • A new report in Nature argues that natural language aptitude is strongly correlated with the ability to learn new programming languages. Math literacy matters far less to students learning new languages [NATURE]

Tools

  • CSS.gg is a minimalistic icon library entirely built in CSS [CSS.GG]
  • cdk8s is a software development framework for defining Kubernetes applications and reusable abstractions using familiar programming languages and rich object-oriented APIs [CDK8S]
  • Neutralino is a portable and lightweight cross platform application development framework [NEUTRALINO]
  • Arkade (ark for short) provides a clean CLI with strongly-typed flags to install charts and apps to your cluster in one command [ARKADE]
  • Monitoror is a unified monitoring wallboard that offers light, ergonomic and reliable monitoring for anything [MONITOROR]
  • 8base brings together the systems your application needs and makes them available through a single GraphQL API endpoint [8BASE]
  • Nots.io is a documentation tool for development teams that lets you access docs from your code [NOTS.IO]
Never miss the big news

Every week, our team will send you three of the most important stories for developers, including our analysis of why they matter. Software development changes fast, but src is your secret weapon to stay up to date in the developer world.

Featured articles
Made with by Software. Read more about our mission.