Behind the Build: Erai Anbu S on Headless Browser Service

Every developer is unique, but the majority of them share a common trait: they are curious problem-solvers driven by a genuine passion for creating meticulously crafted solutions. In this no-nonsense Q&A series, we interview our in-house engineering team to gain insight into how they go about solving problems for fellow developers. The series aims to not only bring you educational content but also foster transparency between our development team and users like you, establishing a stronger connection.

In this Q&A article, we delve into the world of SmartBrowz, a headless browser service, by interviewing Erai Anbu S, an Engineering Lead involved in its development. This interview explores the inspiration behind SmartBrowz, its unique offerings, and the impact it has on developers and businesses.

Let's meet the dev!

Featured here is Erai Anbu S

Erai Anbu S is an Engineering Lead who is passionate about crafting innovative architectural solutions for complex engineering problems. With a versatile background spanning roles in DevOps, Performance and Scalability, Research and Development, and Site Reliability Engineering, he has played a significant role in the development of SmartBrowz.

Q 1: What is the inspiration behind SmartBrowz? Who is it designed for?

Anbu:  SmartBrowz was inspired by customer queries from those using Catalyst serverless functions and were encountering issues with running Puppeteer functions. This prompted us to delve into the world of Puppeteer and headless browsers. Running a headless browser on a server is a complex undertaking. 

• It requires the installation of numerous system libraries, including fonts, emojis, and language support, to ensure smooth operation. 
• Browsers are notorious for their memory-related challenges, making it impractical to run them directly on your application server. Instead, they should be executed within an isolated environment to safeguard the application server's integrity.

It was during this process that we conceived the idea of creating a service that would handle all the intricacies involved, such as managing dependencies, isolating the browser, and overall browser management. Initially, it started as a simple headless browser solution, but it has since evolved into a comprehensive platform offering a wide array of browser-related solutions.

Q 2:  What sets SmartBrowz apart from other headless browser services in the market, and how do you hope developers benefit from it?

Anbu: What sets SmartBrowz apart from other providers is its comprehensive approach to all browser-related functionalities. While many providers tend to simply provide a direct browser or concentrate on a single aspect of browser functionality, SmartBrowz offers a broad spectrum of features by abstracting these browser functions and direct headless browsers as well. This abstraction minimizes browser maintenance, setting it apart from the more specialized solutions provided by others.

Q 3: Can you share a few use cases where SmartBrowz's features and capabilities could offer a competitive advantage or drive innovation?

Anbu: Each of Smartbrowz's features is designed to address various business and engineering challenges, providing innovation to their business. Here are the core use cases and business problems Smartbrowz is built to solve.

1. Testing and Web Automation: Businesses can conduct website testing within the CI/CD pipeline and automate their daily tasks, such as data entry and extraction.
2. Convert anything into APIs: With SmartBrowz's Browser Logic, you can develop a serverless function to seamlessly transform any webpage into a RESTful format. This capability proves valuable for accessing both legacy applications lacking REST APIs and modern applications that do not expose REST APIs.
3. PDF or Screenshot Conversion: Businesses can leverage PDF or Screenshot conversion to produce high-quality PDFs and images. This versatile capability finds applications in various scenarios, such as generating invoices for e-commerce transactions, creating reports and dashboards with images and charts for applications, and issuing certificates. Offering portable documents within applications facilitates data sharing among users and enhances accessibility.
4. Data Enrichment: SmartBrowz's Dataverse feature assists businesses in enhancing their lead data, enabling sales applications to incorporate automated lead enrichment features, thereby reducing the need for manual tasks for end users.

Q 4: What level of control and flexibility does SmartBrowz offer to developers?

Anbu: SmartBrowz offers a range of features, each with its own degree of control and flexibility. Developers can easily implement these features with minimal configuration.

Furthermore, when working directly with the browsers, developers have complete flexibility and control over them. They simply need to integrate our remote browser endpoint into their code, without the need for any further code adjustments. This ensures they maintain full autonomy and authority over their code.

We have abstracted common browser functionalities into features like 'Browser Logic', 'Dataverse', and 'PDF Conversion'. For instance, if you want to convert a webpage to a PDF, instead of writing extensive code, you can achieve it using our SDK or  with a simple API call in SmartBrowz.
  

Q 5: How could developers or businesses benefit by adopting a tool like SmartBrowz? Does it impact their productivity and efficiency? 

Anbu: Setting up browsers in a server environment is a complex task that involves maintenance, version management, and continuous monitoring challenges. Adopting SmartBrowz addresses all these issues, allowing businesses to focus on their core operations without worrying about browser maintenance.

Our built-in solutions like Browser Logic, PDF Conversion and Dataverse further simplify their code logic. Users only need to make an API call, and SmartBrowz will handle data retrieval using browsers, making the process more efficient for them.

Q 6: Hypothetical scenario: Say you are a developer looking for a headless browser service. What important questions would you ask yourself to evaluate if the solution is right for you and your project?

Anbu:  It's crucial to assess and select the appropriate system based on the specific use case. If I were a developer, I would take the following factors into consideration when choosing a headless browser:

• The ease of transitioning from my current native system to this remote browser.
• Whether the system manages version updates and provides failover capabilities.
• Whether the system offers isolated and fresh browsers for each connection.
• The system's ability to automatically scale as the number of connections increases.
• Support for multiple browsers, IP rotation, and session recording.

Q 7: Building software professionally is not only about shipping quickly. It’s also about optimizing for scalability and security. What’s your take on this and how does SmartBrowz handle it?

Anbu: Scalability and security hold significant importance for any product. As a developer platform, we are responsible for running user programs, including resource-intensive applications like browsers, on our system. These user-executed programs can introduce security vulnerabilities. Hosting them in a shared environment would present challenges related to both scalability and security.

To address these concerns, we've established an architecture where all user-related programs and browsers run within dedicated and isolated environments which will ensure both scalability and security.

Q 8: Could you share an exciting use case where SmartBrowz has demonstrated its potential?

Anbu: One particularly compelling business use case involved a client in the digital marketing sector who was looking for an efficient solution to generate and manage leads. SmartBrowz provided an automation solution that streamlined the process of gathering leads from different online sources and seamlessly imported these leads into their Zoho CRM system. 

Another notable use case involved automating a personal task related to securing tickets for a highly anticipated movie release. People were constantly refreshing the movie website in hopes of securing tickets. I witnessed someone using SmartBrowz Headless to automate this manual process. The program monitored the movie site for ticket availability, and when tickets became available, it sent notifications and successfully booked tickets. Although this example is not business-related, it illustrates how SmartBrowz can be applied to automate various browser tasks in your daily routines, making your life more efficient.

Q 9: What are the future plans and enhancements for SmartBrowz that will further benefit senior developers and decision-makers?

Anbu: We have an extensive roadmap for SmartBrowz, with our first priority being the introduction of auto-scalable browser instances, designed to assist QA and web scraping developers. Following this, we have plans to incorporate additional features, including video recording, support for multiple browsers for cross-browser testing, private network testing, and more.

For web scraping, we're looking into adding no-code crawlers and IP rotation features. We're convinced that these enhancements will make it easy for developers to embrace and transition to our system.

Q 10: If there is one thing you'd like every developer or business to know or think about when it comes to browser automation, what would it be?

Anbu: Browser automation is not just about streamlining repetitive tasks. It's about re-imagining what's possible, about innovating workflows, and about harnessing the vast potential of the web in new, transformative ways. With tools like SmartBrowz, the only limit is your imagination.
 

We hope you enjoyed reading this interview where Erai Anbu S shed light on the true potential of browser automation to revolutionize the way we approach web tasks. With SmartBrowz's user-friendly interface and comprehensive capabilities, it's clear that streamlining web automation for diverse use cases is within reach for everyone.
 
We can't wait to share more behind-the-scenes insights with you soon. Stay tuned!