List Documents

Query and list documents with filtering, pagination, sorting, and field projection

GET /{collection}

List Documents

cURL

curl --request GET \
  --url https://api.example.com/{collection} \
  --header 'Authorization: <authorization>'

200

{
  "data": [
    {
      "_id": "<string>",
      "...": "<any>"
    }
  ],
  "pagination": {
    "cursor": "<string>",
    "has_more": true,
    "total": 123
  }
}

Overview

Retrieve a paginated list of documents from a collection with support for:

• Filtering - Query documents by field values
• Cursor-based pagination - Efficient pagination for large datasets
• Sorting - Order results by one or more fields
• Field projection - Select specific fields to return
• RBAC filtering - Automatic query filtering based on user permissions
• Field masking - Sensitive data protection

Request

Path Parameters

collection · string · required

The name of the collection to query

Query Parameters

_limit · integer · default: "20"

Maximum number of documents to return. Cannot exceed max_result_size (default 10000)

_cursor · string

Pagination cursor from previous response to fetch next page. Base64-encoded cursor data.

_sort · string · default: "_id"

Comma-separated list of fields to sort by. Prefix with - for descending order.

Examples:

• created_at - Sort by created_at ascending
• -created_at - Sort by created_at descending
• status,-created_at - Sort by status ascending, then created_at descending

_fields · string

Comma-separated list of fields to include in response. _id is always included.

Example: name,email,created_at

{field} · any

Filter by any field in your schema. Supports:

• Simple equality: ?status=active
• JSON values for complex types: ?age={"$gt":18}
• Multiple fields: ?status=active&role=admin

Headers

Authorization · string · required

Bearer token for authentication

Response

data · array

Array of documents matching the query

_id · string

Document unique identifier

... · any

Other fields from the document, filtered by field-level permissions

pagination · object

Pagination metadata

cursor · string

Cursor for fetching the next page. Empty if no more results.

has_more · boolean

Whether more results are available

total · integer

Total count of documents matching the filter (only included on first page)

Examples

Basic List

curl -X GET "https://api.example.com/users?_limit=10" \
  -H "Authorization: Bearer YOUR_TOKEN"

With Filtering

curl -X GET "https://api.example.com/users?status=active&role=admin" \
  -H "Authorization: Bearer YOUR_TOKEN"

With Sorting

curl -X GET "https://api.example.com/users?_sort=-created_at&_limit=20" \
  -H "Authorization: Bearer YOUR_TOKEN"

With Field Projection

curl -X GET "https://api.example.com/users?_fields=name,email,role" \
  -H "Authorization: Bearer YOUR_TOKEN"

Pagination (Next Page)

curl -X GET "https://api.example.com/users?_cursor=eyJfaWQiOiI1MDdmMWY3N2JjZjg2Y2Q3OTk0MzkwMTEifQ&_limit=20" \
  -H "Authorization: Bearer YOUR_TOKEN"

Complex Filter with JSON

curl -X GET "https://api.example.com/users?age=%7B%22%24gt%22%3A18%7D&status=active" \
  -H "Authorization: Bearer YOUR_TOKEN"

Response Example

{
  "data": [
    {
      "_id": "507f1f77bcf86cd799439011",
      "name": "John Doe",
      "email": "[e-mail]",
      "role": "admin",
      "status": "active",
      "created_at": "2024-03-15T10:30:00Z"
    },
    {
      "_id": "507f1f77bcf86cd799439012",
      "name": "Jane Smith",
      "email": "[e-mail]",
      "role": "member",
      "status": "active",
      "created_at": "2024-03-14T09:20:00Z"
    }
  ],
  "pagination": {
    "cursor": "eyJfaWQiOiI1MDdmMWY3N2JjZjg2Y2Q3OTk0MzkwMTIifQ",
    "has_more": true,
    "total": 157
  }
}

Implementation Details

MongoDB Query Pattern

The handler uses MongoDB’s find operation with cursor-based pagination:

// Build combined filter from user query + RBAC permissions
combinedFilter := bson.M{
    "$and": []bson.M{userFilter, rbacFilter},
}

// Apply cursor for pagination
if cursor != "" {
    cursorFilter := buildCursorFilter(cursorData, sortFields)
    combinedFilter = combineFilters(combinedFilter, cursorFilter)
}

// Execute query with limit+1 to detect hasMore
options := options.Find().
    SetLimit(int64(limit + 1)).
    SetSort(buildSort(sortFields)).
    SetProjection(buildProjection(fields))

cursor, err := collection.Find(ctx, combinedFilter, options)

Workflow

1. Extract collection name from URL path (handlers_query.go:146)
2. Authenticate request and get auth context (handlers_query.go:153)
3. Validate collection exists in schema (handlers_query.go:160)
4. Check RBAC read permission (handlers_query.go:169)
5. Parse query parameters into filter, sort, limit, cursor, fields (handlers_query.go:178)
6. Get RBAC query filter based on user’s roles (handlers_query.go:186)
7. Combine user filter with RBAC filter using $and (handlers_query.go:191)
8. Apply cursor-based pagination if cursor provided (handlers_query.go:194)
9. Build MongoDB sort specification (handlers_query.go:207)
10. Execute query with limit+1 to detect hasMore (handlers_query.go:229)
11. Calculate computed fields for each document (handlers_query.go:246)
12. Apply field policy to hide denied fields (handlers_query.go:247)
13. Apply field masking for sensitive data (handlers_query.go:248)
14. Get total count (only on first page, can be expensive) (handlers_query.go:256)
15. Build next cursor from last document (handlers_query.go:264)
16. Return paginated results (handlers_query.go:285)

Cursor-Based Pagination

The system uses efficient cursor-based pagination instead of offset-based pagination:

// Cursor contains the last document's sort values
type CursorData struct {
    ID         string                 `json:"_id"`
    SortValues map[string]interface{} `json:"sort_values,omitempty"`
}

// Encode cursor from last document
func EncodeCursor(lastDoc map[string]interface{}, sortFields []string) string {
    cursor := CursorData{
        ID: lastDoc["_id"],
        SortValues: make(map[string]interface{}),
    }
    for _, field := range sortFields {
        fieldName := strings.TrimPrefix(field, "-")
        if val, ok := lastDoc[fieldName]; ok {
            cursor.SortValues[fieldName] = val
        }
    }
    data, _ := json.Marshal(cursor)
    return base64.URLEncoding.EncodeToString(data)
}

RBAC Query Filtering

The system automatically filters queries based on user permissions:

// Get RBAC filter based on user's roles and collection policies
rbacFilter, err := h.rbac.GetQueryFilter(authCtx, collection, ActionRead)

// Example RBAC filters:
// - Tenant isolation: {"company_id": user.tenant_id}
// - Owner-only: {"created_by": user.id}
// - Department access: {"department": {"$in": user.departments}}
// - Hierarchical: {"owner_id": {"$in": user.subordinates}}

Field Projection

Optimize bandwidth by selecting specific fields:

if len(fields) > 0 {
    projection := make(bson.M)
    for _, field := range fields {
        projection[field] = 1
    }
    projection["_id"] = 1 // Always include _id for cursor
    findOpts.Projection = projection
}

Error Responses

400 Bad Request

{
  "error": "Invalid cursor"
}

401 Unauthorized

{
  "error": "Authentication required"
}

403 Forbidden

{
  "error": "You don't have permission to perform this action",
  "code": "forbidden",
  "details": {
    "action": "read",
    "collection": "users"
  }
}

404 Not Found

{
  "error": "Collection not found",
  "code": "collection_not_found",
  "details": {
    "collection": "invalid_collection"
  }
}

500 Internal Server Error

{
  "error": "Failed to query documents",
  "code": "internal_error",
  "details": {
    "error": "database timeout"
  }
}

Performance Considerations

Indexes

For optimal performance, create indexes on:

• Fields used in filters
• Fields used in sorting
• Compound indexes for multi-field sorts

// Example: Index for sorting by status and created_at
db.users.createIndex({ status: 1, created_at: -1 })

Total Count

The total count is only computed on the first page (when no cursor is provided) because:

• Counting can be expensive for large collections
• Count accuracy decreases with pagination depth
• Most UIs only need “has more” information

If you need accurate total counts, consider using the Count endpoint.

Limit Recommendations

• Default: 20 documents per page
• Maximum: 10,000 (configurable via max_result_size)
• Recommended: 20-100 for web UIs, 100-1000 for bulk operations

Related Endpoints

• Get Document - Retrieve a single document by ID
• Create Document - Create a new document
• Update Document - Modify an existing document
• Delete Document - Remove a document