QueryParamsProcessor

Reusable; allows any controller to handle queryParams (e.g. :index action)

Notes

  • Purpose: Provides consistent and centralized handling of common query parameters for API endpoints. Designed to process parameters for filtering, sorting, limiting, pagination, and more.

  • Filtering:

    • Can specify which columns to include via the fields parameter.

    • Alternatively, columns to exclude can be specified using the exclude parameter.

    • Default Behavior: If neither fields nor exclude is provided, all columns will be selected and returned.

  • Sorting:

    • The sort parameter defines the column by which the data should be sorted.

    • The order parameter defines the direction of the sort (e.g., ascending, descending, newest, oldest).

    • Default behavior: If neither sort nor order is specified, the default sort field is created_at and the default order is desc.

  • Limit & Pagination:

    • The limit parameter allows the user to restrict the number of records returned.

    • Pagination is achieved through either the page or offset parameter.

      • Note: The page method requires a pagination gem (e.g., kaminari, will_paginate).

    • Default Behavior: If neither limit nor page/offset is specified, all records will be returned without pagination.

      • Note: This could result in large datasets being fetched.

  • Additional Query Params:

    • While they're commented out, there are placeholders for potential future implementations of other common query parameters like where, group, having, distinct, include, joins, and select.

    • Security considerations are mentioned for potential pitfalls with some of these methods (e.g., direct use of user input without sanitization).

  • Private Helper Methods:

    • determine_sort_field: Ensures a valid column is chosen for sorting, defaulting to created_at.

    • column_exists?: Checks if a column exists in the model being queried.

    • determine_direction: Deciphers the desired sort direction based on the provided order parameter.

  • Error Handling & Security:

    • There's a noted need for adding error handling to ensure the robustness of the system.

    • It's emphasized that direct user input should be sanitized and validated to avoid potential security risks, such as SQL injection attacks.

  • Performance Considerations:

    • In operations that could return large datasets (e.g., broad selects without limits), there's a cautionary note about potential performance issues, suggesting regular monitoring and the use of tools like rack-mini-profiler for performance checking.

More Notes (using query params for GET requests)

  1. Conventional RESTful Semantics: According to RESTful principles, GET requests should be idempotent, meaning they don't cause side-effects or modify the state. Using query parameters to filter, sort, limit, and paginate on :index and :show actions aligns well with this idea, as you're essentially specifying how to "view" or "retrieve" the data.

  2. Enhances User Experience and Developer Efficiency:

    • Flexibility for Clients: This pattern gives frontend clients (or other services) the flexibility to retrieve data in the way they need without requiring changes on the backend each time.

    • Reduced Endpoints: Instead of creating multiple specialized endpoints for different views of the data, you can adjust the view via query parameters.

  3. Performance Implications:

    • For :show actions, where you're retrieving a single record, the primary utility of query parameters is to filter fields, as sorting, limiting, and paginating don't typically apply. Being selective about the fields returned can improve performance, especially if the model has many attributes or if some attributes are computationally expensive to generate.

    • For :index actions, all the functionalities (filtering, sorting, limiting, and paginating) play a crucial role. Limiting and paginating are particularly important for performance reasons. Without them, you might end up fetching a huge amount of data which could slow down the system.

  4. Security and Data Integrity:

    • Always be cautious about exposing fields, especially sensitive ones. Even if a field is not typically shown in the UI, a savvy user could manually add it to the fields query parameter. Ensure sensitive fields are either not accessible or are explicitly whitelisted/blacklisted.

    • Validating the query parameters as you've done, especially for sorting and field selection, helps mitigate risks of SQL injection.

  5. Maintainability:

    • Using a module (QueryParamsProcessor) to handle this functionality helps maintain separation of concerns, keeping the controllers lean.

    • This pattern is extensible. If in the future you decide to support more advanced querying capabilities, having a dedicated module will make it easier to expand upon.

  6. Documentation and Training:

    • Ensure that this feature is well-documented so frontend developers know how to use it effectively.

    • Since this pattern isn't a Rails default, it's essential that any new team members are made aware of it and trained on how to use and extend it.

  7. Consideration for Other Actions:

    • For actions like :create, :update, or :delete, the primary data interchange should be in the request body, not the query parameters. The semantics of query parameters (optional, not affecting state) are a better fit for read-only operations.

Details about Filtering, Sorting, Limiting, Pagination

FILTER (fields, exclude)

  • Restrictions: whitelisting/blacklisting keys (per model)

    • Array of keys

    • Model file

    • Dynamically whitelist/blacklist based on user / role (e.g. admin vs other)

  • Concerns:

    • Users get access to data they shouldn’t be able to

SORT (sort, order)

  • Restrictions: …minimal

    • Internal business logic or system behavior

      • sorting by created_at/updated_at = could give insights into internal activity or update patterns)

    • Expensive operations

      • Sorting by certain fields could be expensive

  • Concerns:

    • Sorting can potentially be an expensive operation, especially on large datasets

      • index fields that allow sorting

    • SQL attacks

      • Always ensure that sorting parameters are not directly interpolated into SQL queries.

      • Using ActiveRecord's built-in query methods as you've done is a good approach because it provides a level of protection against SQL injection attacks.

LIMIT (limit)

  • Restrictions: max limit

    • (without, a user could potentially request vast number of records, leading to resource exhaustion)

    • **if user specifies higher than limit, just round down to limit

    • **also consider a default limit

  • Concerns:

    • DoS attacks (Denial of Service — via continuously making requests that pull large amounts of data)

PAGINATION (offset/page)

  • Restrictions: max page size

    • **if user specifies higher than limit, just round down to limit

    • **also consider a default page size

    • **offset-based vs cursor-based

  • Concerns:

    • Pagination can potentially be an expensive operation, especially on large datasets

      • index fields that allow pagination

Error handling

  • "Consider adding error handling. For instance, if someone passes a non-integer for limit, it might cause an error. Validating user input before using it would be beneficial."

Avoiding SQL injection attacks

  • Sanitize (user input) & validate

    • **use ActiveRecord's built-in methods to prevent injection

    • (e.g. sort method = “uses AR’s built-in query methods)

Implement in controller (items_controller.rb):

class ItemsController < ApplicationController
  include QueryParamsProcessor

  def index
  
    ...
    # Process query params
    @items = process_query_params(@items)
    ...
    
    render json: @items
  end
end

Logic:

Previous:indexNextOther Information

Last updated