Introduction
This post depicts a general picture of where web development is headed in 2024 with a focus on the Node.js/React ecosystem - and provides an inside-out report on how Refine.js is starting to make an impact as a React-based meta-framework.
Web-based applications have become an integral part of life in the post-COVID era. Work from home, remote work, online education, electronic banking, e-commerce, amenities delivery, etc. – all are facilitated by web applications of some sort. Web applications are software applications that are hosted remotely on a server and made accessible to clients over the Internet. They are used commercially across industries to offer a wide range of services. Common these days for social interaction, communication, learning, workplace activities, and banking transactions, the backbone of web applications is the HTTP/HTTPS protocol.
Typically, web applications follow the server-client model, and HTTP/HTTPS protocol enables communication between the server and the client. The server-client model is followed in the greater part of the worldwide web. Lambda-based serverless web application architectures have also emerged in recent years.
This post gives us an overview of where web application development stands in the Node.js/React ecosystem entering 2024. We briefly revisit what web applications are, a few of their types, and the steps involved in developing web applications. We then elaborate on some important considerations to be taken into account while developing web applications. We relate to the importance of speed and efficiency in designing and rapidly building well-architected commercial web applications using open-source libraries and frameworks. We emphasize with some details that user experience is central to consuming resources availed by web applications and why effective UI designs, as implemented by popular design systems and frameworks like Material UI, Ant Design, and TailwindCSS are key to providing the best user experiences.
We highlight how efficient data management is an integral part of highly scalable web applications and talk about the advantages that frameworks like Next.js and Remix offer by supporting integration with a broader range of backend API providers, UI libraries, and frameworks. We also discuss authentication and authorization as vital components in securing web applications and touch base on industry-standard techniques of implementing accessibility and internationalization to reach a greater audience.
Near the end, we appreciate the role of open source technologies, contributors, and the communities assembled around them, as well as important contributions that shape and direct the worldwide web software industry. Finally, we talk about some of Refine’s key capabilities as an emerging meta-framework that is helping developers quickly build enterprise-grade data-heavy web applications.
What is a Web Application
A web application is an application that runs on a remote server on the Internet, listens for requests, and serves HTML, XML, and JSON documents using HTTP/HTTPS.
Traditionally, a web application meant delivery of a web service composed in a well-structured XML document via the Simple Object Access Protocol (SOAP), and following a set of descriptions specified according to Universal Description, Discovery and Integration (UDDI) standards. A web service is invoked by a client following a Web Services Description Language (WSDL) specified in another XML file.
With the advent of the web browser and JavaScript scripting; web applications emerged to provide interactive experiences to users thanks to AJAX. AJAX or Asynchronous JavaScript and XML is a group of technologies composed of JavaScript, HTML / XML, CSS, and HTTP objects equipped to communicate with a server using the HTTP / HTTPS protocol. AJAX was an effective toolbelt for developing interactive web applications responsive to data transfer.
More recently, the web has been dominated by applications that offer services using resources that are accessible by Web APIs. Web APIs are similar to web services but are more flexible as they don't need SOAP or any WSDL description. They can output JSON objects besides HTML / XML. Most Web APIs today follow a set of flexible design principles related to representational state and transfer called REST conventions. Web APIs that are structured / named consistently according to REST principles are called RESTful Web APIs. Today, RESTful Web APIs make up the interfaces through which majority of web applications provide their services.
Web Application Languages, Stacks And Frameworks
Web applications are being developed in every imaginable language, both old and emergent. Among the top backend languages used for building web applications are JavaScript, Python, Java, TypeScript, C#, PHP, and Ruby.
Stacks dominating web applications in 2023 were MERN (MongoDB, Express, React, and Node.js), Django, Ruby on Rails, and Java Spring.
Among dominant frontend libraries and frameworks are React, Angular.js, Vue.js, Next.js, Gatsby, Remix, and Nuxt.js.
Types of Web Applications
For a developer, web applications are classified in several ways.
Web applications are classified commonly according to the server-client taxonomy: server-side web applications and client-side web applications.
They are also classified according to stack layer segregation, such as backend web applications, frontend applications, and full-stack web applications.
Web applications are also described according to architectural patterns, certain design decisions guided by business requirements, and the degrees of platform support they offer. Things like MVC applications, monolithic applications, microservices, single page applications, static site generators, progressive applications, etc.
Web Application Development Life Cycle
A typical web application development process follows a cycle of steps and certain methodologies.
Steps in Web Application Development
A general outline of steps involved in a web application development life cycle looks like this:
- Requirements gathering and framing feature specifications: can be from clients, user behavior analytics, and feature requests.
- Planning of resources and development phases: includes team members, roadmaps, budget planning, etc.
- Designing a web application: includes wireframing, visual design / prototyping.
- Stack selection: involves choosing software architecture, components, frameworks, and libraries.
- Coding: includes feature implementation, unit testing, and integration testing.
- Testing: includes black box testing, white box testing, and penetration and integration tests.
- Deployment: includes app/service containerization, container orchestration, and running apps in the cloud.
- Maintenance: includes DevOps, monitoring, analysis, and continuous integration and delivery (CI/CD).
Web Application Development Methodologies
An application development methodology refers to a well-established system that organizes a team and activities to facilitate the development of an application within a set timeline and budgetary constraints following a suitable roadmap. Standard methodologies employed in web application development include Waterfall, Kanban, Scrum, and Agile. A given methodology is appropriate according to the application requirements, project size, and team size.
Considerations in Application Development Cycle
Developers need to take into account important considerations while following the steps outlined above. For example, one top goal is to achieve the best development speed and efficiency. Secondly, to design high-quality user experiences and highly performant, flexible UI solutions. Thirdly, to build/use robust data fetching and management mechanisms that support a wide range of backend API integrations. Implementing a state-of-the-art authentication and authorization system is crucial for securing modern web applications. It is also essential to integrate accessibility and internationalization in a web application for greater audience outreach.
The sections below delve into the evolution of recent technologies that address the above considerations.
Pursuing Development Speed and Efficiency
While going through the planning, designing and stack selection phases, we need to keep in mind that we choose the stacks, components and libraries that give optimal development speed and efficiency. This is important because web applications have been at the heart of global communication and participation since COVID, and well-documented and battle tested solutions are already out there in the open. There are now low-code frameworks and backend API service providers like Airtable, Firebase, Supabase, Strapi and more. There are no good reasons for expending time and resources redefining / building things that are already out there.
Using Frameworks
For example, web application frameworks such as Ruby on Rails or Next.js give us the backbone for many of the features and integrations we would have to spend months working on our own. They also have design principles and conventions that enable faster learning and application development.
Low-code frontend frameworks and libraries provide domain-specific solutions and often support integration with external APIs. For example, Clerk offers an advanced authentication and user management solution with a few lines of code. It also has integration support for Next.js, Gatsby, MERN, and Ruby on Rails. So, while contributing to development efficiency by reducing coding time, low code frameworks and libraries offer a range of integration options, if applicable, to be used in conjunction with.
Using Existing Libraries
It is best to research enough to find the correct libraries or packages for a feature and avoid reinventing the wheel. Packages that have reached end-of-life should be avoided, and so should the ones that are poorly documented and supported. Conversely, technologies that provide loosely coupled integration and feature extension/augmentation should be experimented with and considered in the stack.
Carefully choosing libraries with scalability in mind and that offer feature extensions contributes to development speed and efficiency. For example, Yup is an npm library commonly used in React forms. Its API surface best covers form field parsing and validation. It can be integrated with any React form and comes baked in with Formik. Since its API surface is not tightly coupled to any specific library or framework, it can also be used in React Hook Form forms as well as in other frameworks such as Vue.js and Svelte. In other words, loosely coupled libraries like Yup do their own set of things well and can be plugged in where needed -- saving developers time and effort.
Optimizing User Experience and Interface Design
User experience is the single most important thing when a web application interacts with a user. Page performance dictates a user's experience with a web application. Page load times, latency, fluidity across devices, animations, and transitions -- all contribute to page performance and, hence, user experience.
It is vital to reduce overall load times by minimizing file sizes, the number of HTTPS requests, and latency and effectively using preloading, lazy loading, progressive enhancement, and graceful degradation techniques. Designing fluid user interfaces with consistent experiences across devices is another key component for offering an optimal overall user experience. Using unexaggerated, effective CSS animations and transitions to create high-performance styles and interactions contributes to a positive user experience.
Design systems and UI frameworks like Material UI, Ant Design, and TailwindCSS are geared to deliver highly performant UX and UI solutions for component-based markups in modern web applications.
Effective Data Management and API Integration
Data fetching and presentation perhaps make web applications "running". For most commercial web applications, all business logic implemented in the app is related to data capture, storage, retrieval, and mutation. Dominant architectures like the Model-View-Controller are designed particularly around data modeling, storing, querying, mutating, and presenting.
State Management in Frontend Apps
For frontend React applications that consume RESTful APIs, client-side state management has been addressed by packages like XState, MobX, and Redux. These solutions are great for managing component states. In many apps, these state management libraries have also been used to manage data fetched from a server.
Server Data Management
Data fetching makes modern RESTful APIs what they are. Until recently, server-side data fetching and state management inside frontend applications has been handled ambitiously by Redux Thunk, Saga, and similar libraries. They are now addressed by more robust libraries like Tanstack React Query and Redux Toolkit (RTK) Query. These libraries provide powerful APIs for managing data fetching, caching, request state management, and error handling.
Frameworks such as Next.js and Remix allow easy integrations and management of data with backend APIs built with services like Firebase, Supabase, Strapi, etc.
Secure Applications with Authentication and Authorization
RESTful web applications need mechanisms to control access to API resources first by authenticating a consumer and then by applying authorization checks. Authentication and authorization strategies form the core of securing web APIs and are implemented on both the server side and client side. A myriad of systems and related technologies are used for authentication and authorization.
Authentication in Web Applications
Recently, One Time Password (OTP) has gained traction and has supplemented existing password-based auth strategies with username, email, or phone number. The use of JWT and OAuth 2 libraries for implementing third-party authentication and authorization systems is also more common. Such options are easily integrated with Node.js based libraries like Passport.js and come integrated with robust frontend authentication systems like NextAuth.js.
Role-Based Access: Authorization in Web Applications
Access control plays a vital role in web applications. Libraries like Casbin, Cerbos, and AccessControl.js offer solutions dedicated to implementing authorization both in the backend and the front end.
Greater Outreach with Accessibility and Internationalization
Web applications need to address accessibility and internationalization barriers in order to reach more users.
Accessibility in Web Applications
Accessibility considerations extend to low bandwidth, browser support, and device compatibility -- alongside support for users with special needs.
WAI-ARIA practices must be applied to address people with impairment. Feature detection, polyfills, progressive enhancement, and graceful degradation are standard techniques used to make web applications accessible across browsers and devices.
Internationalization of Web Applications
Internationalization of web applications is essential for reaching a greater audience. i18n and similar libraries provide APIs for translating web applications into locales across geographies. Features include linguistic translations, localization of date, time, currency, keyboard, text collation and sorting, and more.
Open Source Libraries, Support and Community
Open-source libraries play influential roles in the choice of technologies and guide developers to pick proper frameworks and solutions for their stack. We must use well-documented and actively maintained libraries with vibrant communities of developers and contributors.
Using open-source tools and frameworks reduces development time and costs, as most open-source software is either entirely free or incurs a cost for additional features on top of a free tier.
Refine.js as a Trending Meta Framework
Refine.js is a React-based open-source meta-framework geared for building data-heavy enterprise applications. It enables the building of robust React apps such as analytics-driven internal tools, dashboards, admin panels, storefronts, and all sorts of resource-based B2B CRUD applications.
Refine's Core is Headless
Refine comes with a headless core package that bakes in authentication, authorization, API data fetching and management, real-time pub/sub, routing, internationalization, notification support, and audit logging. The core is established on a Hooks-based architecture with app concerns segregated into layers of React contexts backed by their respective providers. Each of these layers can be enabled, disabled, configured, or extended in a Refine app.
Refine's core uses state-of-the-art libraries and technologies under the hood: such as React Query for querying and data state management, React Hook Form and Yup for handling form actions, field validations and error handling in RESTful APIs. GraphQL Query Builder and GraphQL Request APIs are used for achieving the same with GraphQL APIs.
Besides building on top of React Query and React Hook Form, Refine augments its APIs with adapters to interface with supplementary packages. For example, Refine's support package for React Hook Form, @refinedev/react-hook-form, is easily hooked to Ant Design and Material UI components via Refine's own adapters for them. This is possible because Refine's core is loosely coupled and -- on top of the core layer -- it offers integration support for a wide range of frontend and backend APIs.
Refine's UX/UI Integrations
Refine supports integration with Next.js and Remix applications by combining their respective router bindings and utilities with Refine's out-of-the-box.
It also integrates, and augments its core set of providers, hooks, and components with packages for Ant Design, Material UI, Chakra UI and Mantine. These integrations make building complex components and user interfaces with each of these UI frameworks extremely easy. For example, in Refine, sophisticated components like editable tables is enabled with a few lines of code thanks to the useEditableTable() hook. Whether you prefer Ant Design, Material UI, Chakra UI, or Mantine, because Refine's adapters handle interfacing, it follows a similar logic.
Apart from these, other UI frameworks such as TailwindCSS and Bootstrap can easily be integrated into a Refine app by installing respective npm packages and their dependencies.
Refine's Backend Integrations
Refine's support packages for backend integrations is wildly extensive. Refine interfaces with backend services with data providers and auth providers. It has data provider packages for REST, Supabase, Strapi, GraphQL, Appwrite, Airtable, Nest.js, and more. Please feel free to find a comprehensive list here.
Refine's Auth Integrations
Refine supports OTP-based authentication besides perpetual password-based ones. It also has packages for various OAuth strategies, including Google, Auth0, Keycloak, Kinde, and Auth.js.
Besides authentication, Refine also supports easy integrations with Casbin, Cerbos, and AccessControl.js with their npm packages.
Refine's Accessibility and Internationalization Support
Refine supports accessibility that comes with UI frameworks that it integrates with. It also supports internationalization with its i18nProvider. Refine's i18nProvider can take any internationalization library and make it available to the app.
Open Source
Being an open-source framework, Refine has a vibrant community of developers and contributors.
Refer to this fully-functional CRM app built with Refine to get a better idea of what's possible with Refine.