3 is authorized

docs/followups/3-is-authorized.md

@@ -0,0 +1,28 @@
+# Follow-ups for branch 3 — is-authorized (review)
+
+## PR description
+
+Idea 2 for a concrete request: partial evaluation with the request and the action entities known, the
+residuals compiled into one query with the three-valued encoding, the decision rebuilt as `cedar-policy`
+does (`docs/plans/3-is-authorized.md`). Tested against `cedar_policy::Authorizer` case by case, and by the
+`sql-is-authorized-drt` target in `cedar-sql-spec`.
+
+## Review findings
+
+The review (against the committed branch, with some forty probe policies) found no semantic divergence in
+the SQL encoding. Its findings, fixed in branch 5 since branch 4 rewrote the compiler in between:
+
+- A request without a context is one with an unknown context in `cedar-policy`; it was treated as empty.
+- Entity references inside JSON-stored sets and records are compared by id under their static type; the
+  loader now checks stored contents against the schema (`load::conforms`).
+- The quoting assumed `standard_conforming_strings`; connections now set it.
+- `hasTag` on an entity type without tags (which the validator types as `false`) was unsupported.
+- The query text grew exponentially with nested `&&`/`||`; branch 4 binds shared operands to `LATERAL`
+  nodes.
+- On the `cedar-sql-spec` side: loader errors other than NUL strings and `Request`/`Tpe` errors after
+  validation are bugs, not skips; `enable_extensions: false` made typed generation abort inputs at the
+  extension-function arms; the recursive hierarchy mode and `like` were not fuzzed.
+
+## Suggested follow-ups
+
+- Probes worth keeping as tests were added to `tests/authorize_pg.rs` and `tests/hierarchy_pg.rs` (branch 5).

docs/plans/3-is-authorized.md

@@ -0,0 +1,58 @@
+# Plan 3 — is-authorized: concrete authorization through one query
+
+## Goal
+
+Idea 2 of the README for a concrete request: typed partial evaluation with the request known and only the
+action entities in the store, the residual policies compiled into one query over the tables of Plan 2, and
+the decision with its determining and erroring policies rebuilt as `cedar-policy`'s authorizer computes them.
+This branch covers the core operators; sets, records beyond `has`/`get`, tags on computed entities and the
+remaining constructs are `Unsupported` until Plan 4, which the differential test in `cedar-sql-spec` treats
+as benign skips.
+
+## Design
+
+- **Residuals** (`src/authorizer.rs`): `concrete_request` turns a `Request` into a `PartialRequest` with both
+  ids and the context known; `action_entities` turns the action entities into `PartialEntities`;
+  `cedar_policy_core::tpe::is_authorized` then folds everything that does not touch entity data. `SqlAuthorizer::
+  new` validates the policies strictly (every residual node is typed, which the compiler relies on).
+- **The encoding** (`src/compile.rs`): `NULL` is a Cedar error. `&&`, `||` and `if` are `CASE` forms that keep
+  short-circuiting (`false && error` is `false`, `error && false` an error, an untaken branch is not evaluated);
+  `+ - *` compute in `numeric` and range-check so no bigint overflow reaches Postgres (which would abort the
+  statement); unary `-` guards `i64::MIN`; `==` on the same representation is `=`, on different entity types or
+  representations it is `false` unless an operand is `NULL`; `<`/`<=` on integers; `like` translates the pattern
+  to `LIKE … ESCAPE '\'` (`*` to `%`, `%`/`_`/`\` escaped); `is` is a constant from the static type; `hasTag`
+  is a guarded `EXISTS` and `getTag` a scalar subquery on the tags table (no row is an error); `in` is
+  `(same type AND same id) OR EXISTS` on the hierarchy table when it holds the closure, else on a per-anchor
+  recursive ancestors CTE (`UNION`, so cycles terminate); `in` on a literal set is the disjunction. Record
+  attributes read the canonical JSON (`-> 'r' -> 'k'`, cast per static type; `has` is `?`).
+- **Roots and CTEs**: a `Compiled` value carries its representation and, for entities fetched from a row, an
+  `Anchor` (root plus attribute path). Roots are the unknown request variables and the entity literals that are
+  dereferenced; a root's CTE selects the id column as `"$id"`, one `LEFT JOIN` per entity-typed attribute path
+  under it (joined on the target's entity id column, so at most one row each), and one `"$v:<path>"` column per
+  referenced attribute path. `getAttr` is that column (`NULL` for a missing row or an absent attribute, both
+  Cedar errors); `hasAttr` is `IS NOT NULL` guarded by the parent's value, so a missing entity is `false` and an
+  errored parent an error; an attribute the schema does not declare is `false`. Generated names longer than 63
+  bytes are shortened with a hash (`ident::shortened`), since Postgres would truncate them silently.
+- **The query**: `WITH [RECURSIVE] <roots>, <ancestors> SELECT <unknown ids>, p0 … pN FROM (SELECT 1 AS base)
+  CROSS JOIN <unknown roots> LEFT JOIN <literal roots> ON TRUE`, so a concrete request yields exactly one row.
+- **Decision** (`CompiledAuthorization::response`): satisfied permits and forbids are the ones TPE found `true`
+  plus the residual columns that are `TRUE`; `errors` are TPE's plus the `NULL` columns; `Allow` iff some permit
+  and no forbid; the reason is the satisfied forbids if any, else the satisfied permits.
+
+## Files
+
+`src/{compile,authorizer,error}.rs`, `tests/authorize_pg.rs`, `tests/entities/kitchen_sink.json` (a dangling
+reference), `tests/load_pg.rs`, `docs/plans/3-is-authorized.md`.
+
+## Verification
+
+`cargo test --all-features` with `CEDAR_SQL_PG_URL` set: `tests/authorize_pg.rs` authorizes every case through
+SQL and through `cedar_policy::Authorizer` and requires the same decision, reason set and error set — attribute
+chains over existing, absent and dangling references, literal roots, records, `like`, `is`, arithmetic at the
+overflow boundary, short-circuiting with an erroring operand on each side, `in` on both hierarchy modes, tags,
+and the decision rules (forbid overriding, erroring permits and forbids, folded policies next to residual ones).
+Then `cargo clippy --all-targets --all-features -- -D warnings` and `cargo fmt --all --check`.
+
+## History
+
+New; the third `cedar-sql` branch. The differential fuzz target is `cedar-sql-spec` branch `cedar-sql-3-is-authorized`.

src/authorizer.rs

@@ -1,4 +1,261 @@
-//! The public entry points.
+//! The public entry points: authorizing a request against the database.
 //!
-//! Plan 3 adds `SqlAuthorizer` with `compile_request` and `is_authorized`;
-//! Plan 5 adds `query` for partial requests.
+//! [`SqlAuthorizer::is_authorized`] runs typed partial evaluation with the
+//! request (all of it known) and the action entities as the only known
+//! entities, so that every policy either folds to `true`, `false` or an error,
+//! or leaves a residual that references entity data; the residuals become one
+//! query (see [`crate::compile`]) whose row yields each residual policy's
+//! three-valued outcome. The decision, the determining policies and the
+//! erroring policies are then computed as `cedar-policy`'s authorizer does:
+//! `Allow` iff some permit is satisfied and no forbid is, the determining
+//! policies are the satisfied forbids if any else the satisfied permits, and
+//! the erroring policies are reported but do not affect the decision.
+
+use std::collections::{BTreeMap, BTreeSet};
+use std::sync::Arc;
+
+use cedar_policy::{
+    Decision, Entities, PolicyId, PolicySet, Request, Schema, ValidationMode, Validator,
+};
+use cedar_policy_core::ast::{Context, Effect, Entity, EntityUIDEntry};
+use cedar_policy_core::tpe;
+use cedar_policy_core::tpe::entities::{PartialEntities, PartialEntity};
+use cedar_policy_core::tpe::request::{PartialEntityUID, PartialRequest};
+
+use crate::backend::{Backend, Row, SqlValue};
+use crate::compile::Compiler;
+use crate::config::DatabaseConfiguration;
+use crate::dialect::Dialect;
+use crate::{Error, Result};
+
+/// An authorization response.
+#[derive(Clone, Debug, PartialEq, Eq)]
+pub struct Response {
+    /// The decision.
+    pub decision: Decision,
+    /// The determining policies.
+    pub reason: BTreeSet<PolicyId>,
+    /// The policies whose evaluation errored.
+    pub errors: BTreeSet<PolicyId>,
+}
+
+/// The outcome of partial evaluation and compilation for one request.
+#[derive(Clone, Debug)]
+pub struct CompiledAuthorization {
+    /// The query, when some policies are residual; `None` when partial
+    /// evaluation decided every policy.
+    pub sql: Option<String>,
+    /// The residual policies, in the order of the query's policy columns.
+    pub residuals: Vec<(PolicyId, Effect)>,
+    /// The unknown request variables whose ids the query selects first.
+    pub unknown_roots: usize,
+    true_permits: BTreeSet<PolicyId>,
+    true_forbids: BTreeSet<PolicyId>,
+    errors: BTreeSet<PolicyId>,
+}
+
+impl CompiledAuthorization {
+    /// The response for one result row (`None` when there is no query).
+    pub fn response(&self, row: Option<&Row>) -> Result<Response> {
+        let mut permits = self.true_permits.clone();
+        let mut forbids = self.true_forbids.clone();
+        let mut errors = self.errors.clone();
+        if let Some(row) = row {
+            if row.len() != self.unknown_roots + self.residuals.len() {
+                return Err(Error::Query(format!(
+                    "expected {} columns, got {}",
+                    self.unknown_roots + self.residuals.len(),
+                    row.len()
+                )));
+            }
+            for ((id, effect), value) in self.residuals.iter().zip(&row[self.unknown_roots..]) {
+                match value {
+                    SqlValue::Bool(true) => {
+                        match effect {
+                            Effect::Permit => permits.insert(id.clone()),
+                            Effect::Forbid => forbids.insert(id.clone()),
+                        };
+                    }
+                    SqlValue::Bool(false) => {}
+                    SqlValue::Null => {
+                        errors.insert(id.clone());
+                    }
+                    other => {
+                        return Err(Error::Query(format!(
+                            "policy {id} yielded {other:?}, not a boolean"
+                        )));
+                    }
+                }
+            }
+        } else if !self.residuals.is_empty() {
+            return Err(Error::Query(
+                "a row is needed for the residual policies".into(),
+            ));
+        }
+        let decision = if !forbids.is_empty() || permits.is_empty() {
+            Decision::Deny
+        } else {
+            Decision::Allow
+        };
+        let reason = if forbids.is_empty() { permits } else { forbids };
+        Ok(Response {
+            decision,
+            reason,
+            errors,
+        })
+    }
+}
+
+/// Authorizes requests for one schema, database configuration and policy set.
+pub struct SqlAuthorizer<'a> {
+    schema: &'a Schema,
+    config: &'a DatabaseConfiguration,
+    dialect: &'a dyn Dialect,
+    policies: PolicySet,
+}
+
+impl<'a> SqlAuthorizer<'a> {
+    /// Validates `policies` strictly against `schema`, which compilation
+    /// relies on (every residual is typed).
+    pub fn new(
+        schema: &'a Schema,
+        config: &'a DatabaseConfiguration,
+        dialect: &'a dyn Dialect,
+        policies: &PolicySet,
+    ) -> Result<Self> {
+        let result = Validator::new(schema.clone()).validate(policies, ValidationMode::Strict);
+        if !result.validation_passed() {
+            return Err(Error::Validation(
+                result
+                    .validation_errors()
+                    .map(ToString::to_string)
+                    .collect::<Vec<_>>()
+                    .join("; "),
+            ));
+        }
+        Ok(Self {
+            schema,
+            config,
+            dialect,
+            policies: policies.clone(),
+        })
+    }
+
+    /// Partially evaluates the policies for `request` over `entities` (the
+    /// action entities, typically) and compiles the residuals.
+    pub fn compile(
+        &self,
+        request: &PartialRequest,
+        entities: &PartialEntities,
+    ) -> Result<CompiledAuthorization> {
+        let schema = self.schema.as_ref();
+        let response = tpe::is_authorized(self.policies.as_ref(), request, entities, schema)
+            .map_err(|e| Error::Tpe(e.to_string()))?;
+        let ids = |policies: Box<dyn Iterator<Item = &tpe::response::ResidualPolicy> + '_>| {
+            policies
+                .map(|p| PolicyId::new(p.get_policy_id()))
+                .collect::<BTreeSet<_>>()
+        };
+        let true_permits = ids(Box::new(response.true_permits()));
+        let true_forbids = ids(Box::new(response.true_forbids()));
+        let mut errors = ids(Box::new(response.error_permits()));
+        errors.extend(ids(Box::new(response.error_forbids())));
+        let mut residuals: Vec<_> = response
+            .residual_permits()
+            .chain(response.residual_forbids())
+            .collect();
+        residuals.sort_by_key(|p| p.get_policy_id().to_string());
+        if residuals.is_empty() {
+            return Ok(CompiledAuthorization {
+                sql: None,
+                residuals: Vec::new(),
+                unknown_roots: 0,
+                true_permits,
+                true_forbids,
+                errors,
+            });
+        }
+        let mut compiler = Compiler::new(self.config, schema, self.dialect, request);
+        let mut columns = Vec::new();
+        let mut order = Vec::new();
+        for (i, policy) in residuals.iter().enumerate() {
+            let condition = compiler.condition(&policy.get_residual())?;
+            columns.push((format!("p{i}"), condition));
+            order.push((PolicyId::new(policy.get_policy_id()), policy.get_effect()));
+        }
+        let sql = compiler.render(&columns)?;
+        Ok(CompiledAuthorization {
+            sql: Some(sql),
+            residuals: order,
+            unknown_roots: compiler.unknown_roots().len(),
+            true_permits,
+            true_forbids,
+            errors,
+        })
+    }
+
+    /// Authorizes a concrete `request`, with the entity data in the database
+    /// behind `db` and the action entities in `actions`.
+    pub fn is_authorized(
+        &self,
+        db: &mut dyn Backend,
+        request: &Request,
+        actions: &Entities,
+    ) -> Result<Response> {
+        let request = concrete_request(request, self.schema)?;
+        let entities = action_entities(actions, self.schema)?;
+        let compiled = self.compile(&request, &entities)?;
+        let Some(sql) = &compiled.sql else {
+            return compiled.response(None);
+        };
+        let rows = db.query(sql)?;
+        let [row] = rows.as_slice() else {
+            return Err(Error::Query(format!(
+                "expected exactly one row, got {}",
+                rows.len()
+            )));
+        };
+        compiled.response(Some(row))
+    }
+}
+
+/// The partial request with everything known.
+pub fn concrete_request(request: &Request, schema: &Schema) -> Result<PartialRequest> {
+    let core = request.as_ref();
+    let known = |entry: &EntityUIDEntry, what: &str| match entry {
+        EntityUIDEntry::Known { euid, .. } => Ok(PartialEntityUID {
+            ty: euid.entity_type().clone(),
+            eid: Some(euid.eid().clone()),
+        }),
+        EntityUIDEntry::Unknown { .. } => Err(Error::Request(format!("the {what} is unknown"))),
+    };
+    let principal = known(core.principal(), "principal")?;
+    let resource = known(core.resource(), "resource")?;
+    let action = match core.action() {
+        EntityUIDEntry::Known { euid, .. } => euid.as_ref().clone(),
+        EntityUIDEntry::Unknown { .. } => {
+            return Err(Error::Request("the action is unknown".into()));
+        }
+    };
+    let context = match core.context() {
+        Some(Context::Value(values)) => Some(values.clone()),
+        Some(Context::RestrictedResidual(_)) => {
+            return Err(Error::Unsupported("a partially unknown context"));
+        }
+        None => Some(Arc::new(BTreeMap::new())),
+    };
+    PartialRequest::new(principal, action, resource, context, schema.as_ref())
+        .map_err(|e| Error::Request(e.to_string()))
+}
+
+/// The action entities of `entities` as known partial entities.
+pub fn action_entities(entities: &Entities, schema: &Schema) -> Result<PartialEntities> {
+    let actions = entities
+        .as_ref()
+        .iter()
+        .filter(|e| e.uid().entity_type().is_action())
+        .map(|e: &Entity| PartialEntity::try_from(e.clone()).map_err(|e| Error::Tpe(e.to_string())))
+        .collect::<Result<Vec<_>>>()?;
+    PartialEntities::from_entities(actions.into_iter(), schema.as_ref())
+        .map_err(|e| Error::Tpe(e.to_string()))
+}