Discover articles, books and research

With multiple tech giants offering various cloud based services at competitive prices, lots of companies are shifting their infrastructure to the cloud. This has resulted in huge gains for companies, as they get to focus on their business rather than infrastructure management. In recent times, a new architectural design pattern has emerged, born out of cloud native. This new architecture offers a pay as per use model, allowing companies to spend only as per their requirements. This architecture also allows for infinite scaling, both scale-up and scale-down. Modern workloads on applications vary drastically, and have become increasingly difficult to predict. This leads to the conundrum of resource management, allocating excessive resources in order to prepare for a surge in visitors causes wastage in terms of unused compute resources, whereas allocating a smaller amount of resources can be even more dangerous, leading to loss of potential customers. Serverless architecture allows for a granular level of control over resources, with auto-scaling, thus ensuring that we pay for only what we use, along with scale-up and scale-down depending on workloads. This project aims to compare the various serverless offerings, their pros and cons, and develop a standardised application using the best of services and practices to ensure that the application is fully scalable, and economical such that it serves as a benchmark and proof of concept for all such applications. For the purpose of this dissertation, we would be building a recipe sharing website, with multiple features like signing up, OAuth login, recipe creation, update, liking, commenting as well as reporting a recipe if it violates our terms. An admin panel allowing for managing complaints, warning users, banning users and archiving recipes would also be supported. A robust search and filtering mechanism would allow any user, even without authentication to browse recipes, and liking or commenting on any would require signing in. Apart from these, our application would automatically be able to estimate the calories of a recipe based on the ingredient list, for which we use a third party API. Overall, this project aims to provide a pleasant user experience for any consumer wishing to browse recipes or share their creations with the world. The future scope of this project would involve a donations page, allowing for users to donate to various charities via the website and help in the maintenance of the application.

Serverless computing has emerged as a compelling paradigm for scalable and cost-efficient cloud execution, yet its applicability to data-intensive workloads remains an active area of research. This thesis investigates four key research questions related to the use of serverless platforms for such workloads, focusing on representative ML-driven use cases: vector embedding search, distributed DNN inference and traffic forecasting. Through a systems and empirical analysis, the work explores limitations and optimizations for serverless execution in these contexts, providing insights into both performance and cost efficiency. The thesis first examines the feasibility of highly parallelized, hybrid vector similarity search in a serverless environment. A quantization-based indexing technique is proposed, alongside a hierarchical invocation mechanism which enables large-scale synchronous parallelism while mitigating communication overheads. The thesis then investigates the challenges of fully serverless distributed deep learning inference, particularly the constraints imposed by the lack of direct inter-function communication capabilities. Communication channels leveraging publish-subscribe/message queueing services, as well as object storage, are explored, with a cost-performance analysis guiding configuration choices. Finally, an end-to-end ML pipeline is built as a hybrid serverless/server-ful architecture, with several domain-specific enhancements to improve the predictive performance of the models in practice. The thesis demonstrates that serverless computing, when optimized, can provide a costeffective and performant solution for data-intensive and ML-driven workloads, particularly for sporadic use cases. While inherent constraints such as stateless execution and limited inter-function communication impose challenges, novel strategies - including structured invocation patterns, optimized data retention, and hybrid execution models - can significantly improve efficiency. The thesis contributes a systematic examination of these trade-offs, offering insights into the evolving role of serverless computing for data-intensive and MLdriven applications