Did the fight for an open web create a facial recognition nightmare?
A recent exposé from the New York Times revealed the startling privacy concerns surrounding Clearview AI, a secretive startup that uses facial recognition to match photos of unknown people to their online images.
The company owes much of its success to the fervent defense of web scraping, which has led to unintended consequences in security, privacy, and culture.
How does it work? Clearview AI has spent years compiling its database of billions of images scraped from profiles on Facebook, YouTube, Twitter, Instagram, Venmo, and other social websites.
Companies and government agencies using Clearview AI can upload a picture of any face into its system and discover any matching photos of that person across the internet. Matches are displayed alongside links to related social media profiles, making it easy to identify and locate people.
Is it legal? Probably. While such rampant data scraping violates the terms of service for most targeted websites, recent court cases have upheld the legality of automated, large-scale web scraping.
In late 2019, a court determined in hiQ Labs vs LinkedIn that accessing publicly available information in an automated way does not violate the Computer Fraud and Abuse Act.
Many saw this as a huge win for developers and a more open web. As we noted:
"While many companies hope to protect the data they have collected on their platforms, others are looking to leverage that data to build new platforms. With greater freedom for web scraping, developers will have access to many new and legal data sources."
Unintended consequences. Web scraping has certainly had a beneficial impact on developers building innovative and powerful apps, platforms, and tools. But combining the increasingly nefarious use of facial recognition with the legality of scraping has unintentionally created a world in which beasts like Clearview AI can thrive.
More newbies than ever
DEV, an online platform where developers can share original content, acquired CodeNewbie, a community for new developers who are learning to code.
Background: DEV describes itself as a "platform where software developers write articles, take part in discussions, and build their professional profiles." The DEV platform, which is entirely open source, emphasizes transparency and openness.
With nearly 6 million unique visitors per month and upwards of 250,000 registered users, DEV is building an influential global community. At the end of 2019, DEV announced that they had raised their $11.5M Series A round of funding.
A rich ecosystem: DEV joins an increasingly diverse ecosystem of developer communities—including sites like Stack Overflow, Hashnode, Reddit, FreeCodeCamp, HackerNews, and HackerNoon.
CodeNewbie, however, caters to developers who are early in their engineering careers. It created a series of popular podcasts for beginners and runs a developer conference, CodeLand.
Finding their voice: CodeNewbie's rise to prominence shows that newbie developers are a fast growing segment in the developer community.
This influx of people learning to code is driven by (1) people interested in becoming software developers and (2) existing workers whose roles are becoming more technical. More jobs today—like data scientists, machine learning engineers, and cybersecurity specialists—are integrated more deeply into development workflows.
Experts project there will be 27.7 million developers in the world by 2023, up from 23 million in 2018. That doesn’t include millions more that code in some capacity: GitHub alone welcomed more than 10 million new developers to its community in the last year and is now home to over 40 million users.
Why newbies matter: They play an outsized role in shaping the future of software development. Among other things, new developers can drive language and framework adoption toward more beginner-friendly developer tools.
As we enter a golden age of software development, expect it to be increasingly shaped by this rapidly expanding community of new developers—and the online platforms they support.
GraphQL takes a big leap forward with MongoDB
MongoDB announced that developers can now interact with their MongoDB databases using the GraphQL API standard. For GraphQL, MongoDB's support signals its growing importance in shaping the future of software development.
What’s GraphQL? It’s an up-and-coming alternative to traditional API standards, like REST and SOAP. From the official documentation: "GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data."
With GraphQL, developers make API requests to a single endpoint, but structure their queries to ask for specific data from any number of resources. The API then compiles and returns data with the same structure as the original request.
GraphQL matters. GraphQL's flexibility makes it easy for developers to request and structure exactly the data they need in just a single trip to the database. While not a silver bullet, it offers developers a new way to build data-driven applications.
And it’s gaining popularity. In 2016, just 5% of developers had ever used GraphQL. In three years, that number has skyrocketed to nearly 38%.
Today GraphQL is used at many big tech companies, like Facebook, GitHub, Pinterest, Coursera, and Shopify.
Now it works with MongoDB. MongoDB’s new features make adding GraphQL support easy for developers.
What’s the impact? MongoDB’s new features are a noteworthy validation of both GraphQL and serverless architecture as valuable development tools. A fusion of these fledgling technologies, with the backing of a big database player, marks a big step forward for the entire ecosystem.
Microsoft gives developers a closer inspection
Microsoft released an open source version of its Application Inspector, a cross-platform tool that engineers can use to understand potential security issues when integrating third-party software in their codebase.
Automated scanning: Developers can run their applications through Microsoft’s Application Inspector, which analyzes the source code and identifies key features of their software.
Developers can "surfac[e] features of interest and other characteristics to answer the question 'what's in it' using static analysis." That makes it ideal for scanning components before integrating them into a codebase or detecting feature level changes.
The Application Inspector then generates a report that identifies "application frameworks, cloud interfaces, cryptography, sensitive data like access keys, personally identifiable information, operating system functions, and security features."
Once a developer understands these features of their code, they can better understand how open source software impacts their products’ functionality and security.
A new angle on the growing security problem: Most applications today include thousands of lines of code written by thousands of other developers. As a result, many tech companies are working to implement automated and more rigorous security guardrails.
Application Inspector follows this trend, but approaches the problem from a slightly different angle. It works as a code profiler, searching for key characteristics in a codebase and empowering developers to decide if its functionality is scoped appropriately.
That makes the Application Inspector more high-level than simple package vulnerability management or safe code analysis—arming developers with richer automatic tooling for their growing security responsibilities.
- A survey of 21,000 developers conducted by CodinGame revealed that Python is the most loved language—favored by 36% of developers—and PHP is the most dreaded language—disliked by 25% of developers. In general, developers reported being happy with their jobs, with the average developer rating their happiness as 7 out of 10 [CODINGAME]
- The developer behind Actix Web, an open source Rust library, decided to abandon the project after conflicts with the community. He argued that being a maintainer of a large open source project is not enjoyable, as he was often subject to hateful or rude messages [THE REGISTER]
- Shopify opened up its new computer science degree program to applications from aspiring developers. Students in the program will gain hands-on experience at Shopify, while receiving an accredited computer science degree from either Carleton University or York University [SHOPIFY]
- OneDev is an open source platform that includes issue tracking, Git management, and pull requests [GITHUB]
- sscaffold is another modern, lightweight css library that builds on the efforts of previous libraries like milligram, skeleton, and normalize [SSCAFFOLD]
- React-email-editor is a drag-n-drop email editor component for React.js [GITHUB]
- Calendarize is a tiny (196B) utility to generate calendar views [GITHUB]
- Pendulum is a Python library that gives you richer functionality when working with datetimes [PENDULUM]
Every week, our team will send you three of the most important stories for developers, including our analysis of why they matter. Software development changes fast, but src is your secret weapon to stay up to date in the developer world.
Today global initiatives on AI are a series of regulatory and ethical gambles—a dangerous, potentially existential game.
Why the Xbox will be Azure’s unlikely hero.
Understanding churn rates can help developers be more productive and write quality code