horae-browser-checks
checks.aarch64-linux.browser
· build #431
· raw
1elastic-arm-976bb8e2: building 91gjbjrsmlfcmng50kr3ji53bpjbzlfc-horae-browser-checks.drv2initdb: warning: enabling "trust" authentication for local connections3initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.4waiting for server to start.... done5server started62026-10-08T02:06:44.340263Z INFO horae: Running migrations...72026-10-08T02:06:44.934191Z INFO horae: Migrations complete.82026-10-08T02:06:45.006821Z INFO horae::seed: Seeding demo data…92026-10-08T02:06:45.068033Z INFO horae::seed: Seed complete.102026-10-08T02:06:45.096436Z INFO horae::seed: time_entries: 10 rows112026-10-08T02:06:45.101681Z INFO horae::seed: clients: 2 rows12PASS: header actions, supported metrics and labelled native budget progress13PASS: design heading, compact row typography and inline project type14PASS: filtered empty state resets search, scope and client15PASS: narrow/short layouts, keyboard row actions and export formats16PASS: loading spend is unavailable until the read completes17PASS: failed spend is unavailable; retry restores real amounts18PASS: configured monthly consumption replaces lifetime spend only after loading19PASS: keyboard-accessible scope breakdown exposes overruns without layout overflow20PASS: failed configured budget never falls back to lifetime figures21PASS: empty-state guidance and action visibility for admin22PASS: empty-state guidance and action visibility for manager23PASS: empty-state guidance and action visibility for member24PASS: shared Menu, Combobox, empty-state and navigation-icon defaults are unchanged25PASS: /clients header at 320px26PASS: /clients header at 390px27PASS: /clients header at 768px28PASS: /clients header at 1280px29PASS: /clients header at 1440px30PASS: /projects header at 320px31PASS: /projects header at 390px32PASS: /projects header at 768px33PASS: /projects header at 1280px34PASS: /projects header at 1440px35PASS: /invoices header at 320px36PASS: /invoices header at 390px37PASS: /invoices header at 768px38PASS: /invoices header at 1280px39PASS: /invoices header at 1440px40PASS: /reports header at 320px41PASS: /reports header at 390px42PASS: /reports header at 768px43PASS: /reports header at 1280px44PASS: /reports header at 1440px45PASS: /admin/users header at 320px46PASS: /admin/users header at 390px47PASS: /admin/users header at 768px48PASS: /admin/users header at 1280px49PASS: /admin/users header at 1440px50PASS: /approvals header at 320px51PASS: /approvals header at 390px52PASS: /approvals header at 768px53PASS: /approvals header at 1280px54PASS: /approvals header at 1440px55PASS: /settings header at 320px56PASS: /settings header at 390px57PASS: /settings header at 768px58PASS: /settings header at 1280px59PASS: /settings header at 1440px60PASS: /admin/importers header at 320px61PASS: /admin/importers header at 390px62PASS: /admin/importers header at 768px63PASS: /admin/importers header at 1280px64PASS: /admin/importers header at 1440px65PASS: /timesheet/week/2026-10-05 header at 320px66PASS: /timesheet/week/2026-10-05 header at 390px67PASS: /timesheet/week/2026-10-05 header at 768px68PASS: /timesheet/week/2026-10-05 header at 1280px69PASS: /timesheet/week/2026-10-05 header at 1440px70PASS: /timesheet/day/2026-10-05 header at 320px71PASS: /timesheet/day/2026-10-05 header at 390px72PASS: /timesheet/day/2026-10-05 header at 768px73PASS: /timesheet/day/2026-10-05 header at 1280px74PASS: /timesheet/day/2026-10-05 header at 1440px75PASS: /timesheet/calendar/2026-10-05 header at 320px76PASS: /timesheet/calendar/2026-10-05 header at 390px77PASS: /timesheet/calendar/2026-10-05 header at 768px78PASS: /timesheet/calendar/2026-10-05 header at 1280px79PASS: /timesheet/calendar/2026-10-05 header at 1440px80PASS: project names and badges stay within aligned columns; scrolling to last-row actions opens editing81PASS: small-screen timesheet pager, date picker and view controls remain usable82PASS: admin subnavigation and its content remain reachable on mobile83PASS: day strip scrolls to Sunday and long entry labels do not cover actions84CHECK: late popover initialization restores missing focus without overriding deliberate focus85PASS: late popover initialization restores missing focus without overriding deliberate focus86CHECK: content reflow preserves an open menu while deliberate scrolling still dismisses it87PASS: content reflow preserves an open menu while deliberate scrolling still dismisses it88CHECK: first and last project-row actions are initially visible and clickable89PASS: first and last project-row actions are initially visible and clickable90CHECK: menu keyboard, disabled items, outside click and repeated opening91PASS: menu keyboard, disabled items, outside click and repeated opening92CHECK: scroll and resize dismiss without leaving stale menus or focus93PASS: scroll and resize dismiss without leaving stale menus or focus94CHECK: short-screen menu scrolls internally and supports reverse Tab and Space95PASS: short-screen menu scrolls internally and supports reverse Tab and Space96CHECK: filter and calendar menus keep their callbacks and survive route changes97PASS: filter and calendar menus keep their callbacks and survive route changes98CHECK: Timesheet keeps the shared calendar and week selection behavior99PASS: Timesheet keeps the shared calendar and week selection behavior100PASS: mobile content width, keyboard toggle, Escape/focus restoration and navigation close101PASS: short mobile viewport reaches account navigation and preserves desktop collapse state102PASS: desktop drag width survives mobile and breakpoint transitions103PASS: keyboard selection, mixed/select-all states and Actions count104PASS: search, status and client changes clear selection105PASS: keyboard selection/confirmation and no document overflow in both themes at three widths106PASS: cancel/focus, failure retry, pending guard and archive/reactivate confirmation107PASS: oversized selection cannot submit and members have no bulk controls108PASS: real atomic archive/reactivate, anonymous/member/inactive denial, manager/admin and safe retries109PASS: bulk success restores focus and recovers the list without stale actions110PASS: bulk pending restores focus and recovers the list without stale actions111PASS: bulk failure restores focus and recovers the list without stale actions112PASS: client activation failure remains visible with form closed, open and cancelled113PASS: successful form save preserves action error; real status retry clears it and refreshes the row114PASS: archive failure survives menu dismissal; shared edit failure preserves input and safely retries115PASS: assignment removal failure remains visible with form closed, open and cancelled116PASS: initial catalog/draft failures show loading and recovery without rendering or saving an empty replacement117PASS: design typography, section spacing, quiet dividers and theme-aware task information118PASS: project types retain native keyboard selection and the designed icon/option-card states119PASS: searchable client selection, pending/error recovery and explicit/inherited currency120PASS: client default-rate currency rejects relabelling, preserves failed input and permits compatible edits121PASS: task and teammate searches handle pending results, keyboard selection, duplicates and retry122PASS: real client, team, task restrictions, billing and invoice defaults survive reload123PASS: all invoice-default errors identify and focus their field, preserve input and recover at three widths124PASS: basic-field rejections retain values, link visible errors and restore focus at three widths125PASS: failed resume and retry preserve the existing draft without a replacement write126PASS: invalid hourly rates and hour/fee budgets retain input and focus their error at three widths127PASS: a stale email configuration rejection can be corrected without sending mail128PASS: task/person rate, cost and budget errors identify their row and preserve other inputs at three widths129PASS: single/monthly fees, empty schedules and individual milestone errors recover without changing another row130PASS: responsive form, conditional panels and Cancel preserve the draft131PASS: edits made while a save is pending are saved after its acknowledgement132PASS: lost acknowledgement retries the same request and preserves newer edits133PASS: real finalized budget reaches the authorized Projects endpoint and display134PASS: a new tagged project reaches reports and both exports; pending/error filters never show stale rows135PASS: project invoice defaults prefill real estimates, stale reviews cannot generate, and editable draft values persist independently136PASS: lost creation response is idempotent and clears the completed draft137PASS: stale tabs cannot overwrite saved input; explicit discard removes only the draft138locator.fill: Timeout 30000ms exceeded.139Call log:140 - waiting for getByLabel('Period from', { exact: true })141 - locator resolved to <input type="date" id="inv-from" placeholder="" class="form-input" data-dioxus-id="84"/>142 - fill("2026-09-01")143 - attempting fill action144 - waiting for element to be visible, enabled and editable145146 at newFeeInvoice (/nix/store/zn2f4srw60ixzmxqx0dgar80gvbmzry4-browser/invoice-preparation.cjs:382:61)147 at async /nix/store/zn2f4srw60ixzmxqx0dgar80gvbmzry4-browser/invoice-preparation.cjs:394:5 {148 name: 'TimeoutError'149}150{151 url: 'http://127.0.0.1:8093/invoices',152 errors: [],153 page: 'locator.ariaSnapshot: Timeout 3000ms exceeded.\n' +154 'Call log:\n' +155 "\x1B[2m - waiting for locator('body')\x1B[22m\n"156}157locator.fill: Timeout 30000ms exceeded.158Call log:159 - waiting for getByLabel('Period from', { exact: true })160 - locator resolved to <input type="date" id="inv-from" placeholder="" class="form-input" data-dioxus-id="84"/>161 - fill("2026-09-01")162 - attempting fill action163 - waiting for element to be visible, enabled and editable164165 at newFeeInvoice (/nix/store/zn2f4srw60ixzmxqx0dgar80gvbmzry4-browser/invoice-preparation.cjs:382:61)166 at async /nix/store/zn2f4srw60ixzmxqx0dgar80gvbmzry4-browser/invoice-preparation.cjs:394:5 {167 name: 'TimeoutError'168}1692026-10-08T02:06:45.269632Z WARN horae: DEV_LOGIN=1: /auth/dev-login signs anyone in as an admin without a password. Use it for local development only.1702026-10-08T02:15:12.725537Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=1.482293674s elapsed_secs=1.4822936740000001 slow_threshold=1s1712026-10-08T02:15:17.019803Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id\n FROM horae_jobs\n WHERE attempts < max_attempts AND NOT cancellation_requested\n AND ((status = 'queued' AND available_at <= now())\n OR (status = 'running' AND lease_until < now()))\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED\n LIMIT 1\n )\n UPDATE horae_jobs j\n SET status = 'running', attempts = j.attempts + 1,\n lease_until = now() + $1::int * interval '1 second', worker_id = $2,\n claim_token = $3,\n started_at = COALESCE(j.started_at, now()), updated_at = now()\n FROM candidate\n WHERE j.id = candidate.id\n RETURNING j.id, j.org_id, j.payload\n" rows_affected=0 rows_returned=0 elapsed=2.91848476s elapsed_secs=2.91848476 slow_threshold=1s1722026-10-08T02:15:18.418983Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=3.058116801s elapsed_secs=3.058116801 slow_threshold=1s1732026-10-08T02:15:21.711505Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="DELETE FROM horae_job_uploads u …" db.statement="\n\nDELETE FROM horae_job_uploads u\n USING horae_jobs j\n WHERE u.job_id = j.id\n AND j.status = 'succeeded'\n AND j.finished_at < now() - interval '1 day'\n" rows_affected=0 rows_returned=0 elapsed=1.665913306s elapsed_secs=1.665913306 slow_threshold=1s1742026-10-08T02:15:23.706861Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=2.535439749s elapsed_secs=2.535439749 slow_threshold=1s1752026-10-08T02:15:26.944320Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id\n FROM horae_jobs\n WHERE attempts < max_attempts AND NOT cancellation_requested\n AND ((status = 'queued' AND available_at <= now())\n OR (status = 'running' AND lease_until < now()))\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED\n LIMIT 1\n )\n UPDATE horae_jobs j\n SET status = 'running', attempts = j.attempts + 1,\n lease_until = now() + $1::int * interval '1 second', worker_id = $2,\n claim_token = $3,\n started_at = COALESCE(j.started_at, now()), updated_at = now()\n FROM candidate\n WHERE j.id = candidate.id\n RETURNING j.id, j.org_id, j.payload\n" rows_affected=0 rows_returned=0 elapsed=2.30882417s elapsed_secs=2.30882417 slow_threshold=1s1762026-10-08T02:15:28.131817Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=1.642449193s elapsed_secs=1.642449193 slow_threshold=1s1772026-10-08T02:15:30.869378Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="DELETE FROM horae_job_uploads u …" db.statement="\n\nDELETE FROM horae_job_uploads u\n USING horae_jobs j\n WHERE u.job_id = j.id\n AND j.status = 'succeeded'\n AND j.finished_at < now() - interval '1 day'\n" rows_affected=0 rows_returned=0 elapsed=1.10031702s elapsed_secs=1.10031702 slow_threshold=1s1782026-10-08T02:15:33.525880Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="DELETE FROM horae_jobs WHERE …" db.statement="\n\nDELETE FROM horae_jobs\n WHERE status IN ('succeeded', 'failed', 'cancelled')\n AND finished_at < now() - interval '30 days'\n" rows_affected=0 rows_returned=0 elapsed=1.104311499s elapsed_secs=1.104311499 slow_threshold=1s1792026-10-08T02:15:35.735983Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=4.417892636s elapsed_secs=4.417892636 slow_threshold=1s1802026-10-08T02:15:35.380825Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="UPDATE horae_jobs SET status …" db.statement="\n\nUPDATE horae_jobs\n SET status = CASE WHEN cancellation_requested THEN 'cancelled' ELSE 'failed' END,\n lease_until = NULL, worker_id = NULL, claim_token = NULL,\n finished_at = now(), updated_at = now(),\n last_error = CASE WHEN cancellation_requested THEN NULL\n ELSE COALESCE(last_error, 'Job attempt limit reached after interruption') END\n WHERE (attempts >= max_attempts OR cancellation_requested)\n AND (status = 'queued' OR (status = 'running' AND lease_until < now()))\n" rows_affected=0 rows_returned=0 elapsed=1.242801501s elapsed_secs=1.242801501 slow_threshold=1s1812026-10-08T02:15:39.666761Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id\n FROM horae_jobs\n WHERE attempts < max_attempts AND NOT cancellation_requested\n AND ((status = 'queued' AND available_at <= now())\n OR (status = 'running' AND lease_until < now()))\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED\n LIMIT 1\n )\n UPDATE horae_jobs j\n SET status = 'running', attempts = j.attempts + 1,\n lease_until = now() + $1::int * interval '1 second', worker_id = $2,\n claim_token = $3,\n started_at = COALESCE(j.started_at, now()), updated_at = now()\n FROM candidate\n WHERE j.id = candidate.id\n RETURNING j.id, j.org_id, j.payload\n" rows_affected=0 rows_returned=0 elapsed=3.422733467s elapsed_secs=3.422733467 slow_threshold=1s1822026-10-08T02:15:40.419356Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=2.026787768s elapsed_secs=2.026787768 slow_threshold=1s1832026-10-08T02:15:45.619923Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=2.328468365s elapsed_secs=2.328468365 slow_threshold=1s1842026-10-08T02:15:49.702375Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id\n FROM horae_jobs\n WHERE attempts < max_attempts AND NOT cancellation_requested\n AND ((status = 'queued' AND available_at <= now())\n OR (status = 'running' AND lease_until < now()))\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED\n LIMIT 1\n )\n UPDATE horae_jobs j\n SET status = 'running', attempts = j.attempts + 1,\n lease_until = now() + $1::int * interval '1 second', worker_id = $2,\n claim_token = $3,\n started_at = COALESCE(j.started_at, now()), updated_at = now()\n FROM candidate\n WHERE j.id = candidate.id\n RETURNING j.id, j.org_id, j.payload\n" rows_affected=0 rows_returned=0 elapsed=3.608141976s elapsed_secs=3.6081419759999998 slow_threshold=1s1852026-10-08T02:15:50.100757Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=1.701662856s elapsed_secs=1.701662856 slow_threshold=1s1862026-10-08T02:15:54.455871Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="DELETE FROM horae_job_uploads u …" db.statement="\n\nDELETE FROM horae_job_uploads u\n USING horae_jobs j\n WHERE u.job_id = j.id\n AND j.status = 'succeeded'\n AND j.finished_at < now() - interval '1 day'\n" rows_affected=0 rows_returned=0 elapsed=1.651911309s elapsed_secs=1.651911309 slow_threshold=1s1872026-10-08T02:15:58.553351Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="UPDATE horae_jobs SET status …" db.statement="\n\nUPDATE horae_jobs\n SET status = CASE WHEN cancellation_requested THEN 'cancelled' ELSE 'failed' END,\n lease_until = NULL, worker_id = NULL, claim_token = NULL,\n finished_at = now(), updated_at = now(),\n last_error = CASE WHEN cancellation_requested THEN NULL\n ELSE COALESCE(last_error, 'Job attempt limit reached after interruption') END\n WHERE (attempts >= max_attempts OR cancellation_requested)\n AND (status = 'queued' OR (status = 'running' AND lease_until < now()))\n" rows_affected=0 rows_returned=0 elapsed=1.0236872s elapsed_secs=1.0236872 slow_threshold=1s1882026-10-08T02:15:59.291302Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=5.532521251s elapsed_secs=5.532521251 slow_threshold=1s1892026-10-08T02:16:02.630624Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH due AS ( …" db.statement="\n\nWITH due AS (\n UPDATE time_entries te\n SET notified_long_running_at = now()\n FROM organizations o\n WHERE o.id = te.org_id\n AND te.is_running = true\n AND te.notified_long_running_at IS NULL\n AND te.started_at < now() - make_interval(mins => o.long_timer_minutes)\n RETURNING te.id, te.org_id, te.user_id, te.project_id, te.task_id,\n te.spent_date, te.minutes, te.billable, te.notes, te.started_at\n )\n SELECT id, org_id, user_id, project_id, task_id,\n spent_date as \"spent_date: chrono::NaiveDate\",\n minutes, billable, notes,\n started_at as \"started_at!: chrono::DateTime<chrono::Utc>\",\n (EXTRACT(EPOCH FROM (now() - started_at)) / 60)::int as \"running_minutes!\"\n FROM due\n" rows_affected=0 rows_returned=0 elapsed=16.72551392s elapsed_secs=16.72551392 slow_threshold=1s1902026-10-08T02:16:05.001405Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=2.364774235s elapsed_secs=2.364774235 slow_threshold=1s1912026-10-08T02:16:05.001362Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="SELECT ps.project_id, ps.org_id FROM …" db.statement="\n\nSELECT ps.project_id, ps.org_id FROM project_settings ps\n JOIN projects p ON p.id = ps.project_id AND p.org_id = ps.org_id\n WHERE ps.alert_enabled AND p.active AND ps.project_id > $1\n ORDER BY ps.project_id LIMIT 100\n" rows_affected=0 rows_returned=0 elapsed=1.401527778s elapsed_secs=1.401527778 slow_threshold=1s1922026-10-08T02:16:04.819080Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id\n FROM horae_jobs\n WHERE attempts < max_attempts AND NOT cancellation_requested\n AND ((status = 'queued' AND available_at <= now())\n OR (status = 'running' AND lease_until < now()))\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED\n LIMIT 1\n )\n UPDATE horae_jobs j\n SET status = 'running', attempts = j.attempts + 1,\n lease_until = now() + $1::int * interval '1 second', worker_id = $2,\n claim_token = $3,\n started_at = COALESCE(j.started_at, now()), updated_at = now()\n FROM candidate\n WHERE j.id = candidate.id\n RETURNING j.id, j.org_id, j.payload\n" rows_affected=0 rows_returned=0 elapsed=5.516441856s elapsed_secs=5.516441856 slow_threshold=1s1932026-10-08T02:16:09.793076Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="DELETE FROM horae_job_uploads u …" db.statement="\n\nDELETE FROM horae_job_uploads u\n USING horae_jobs j\n WHERE u.job_id = j.id\n AND j.status = 'succeeded'\n AND j.finished_at < now() - interval '1 day'\n" rows_affected=0 rows_returned=0 elapsed=1.400579778s elapsed_secs=1.400579778 slow_threshold=1s1942026-10-08T02:16:13.179731Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=5.187092985s elapsed_secs=5.187092985 slow_threshold=1s1952026-10-08T02:16:14.455122Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="UPDATE horae_jobs SET status …" db.statement="\n\nUPDATE horae_jobs\n SET status = CASE WHEN cancellation_requested THEN 'cancelled' ELSE 'failed' END,\n lease_until = NULL, worker_id = NULL, claim_token = NULL,\n finished_at = now(), updated_at = now(),\n last_error = CASE WHEN cancellation_requested THEN NULL\n ELSE COALESCE(last_error, 'Job attempt limit reached after interruption') END\n WHERE (attempts >= max_attempts OR cancellation_requested)\n AND (status = 'queued' OR (status = 'running' AND lease_until < now()))\n" rows_affected=0 rows_returned=0 elapsed=1.39059102s elapsed_secs=1.39059102 slow_threshold=1s1962026-10-08T02:16:20.576337Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=4.019030304s elapsed_secs=4.019030304 slow_threshold=1s1972026-10-08T02:16:22.021011Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id\n FROM horae_jobs\n WHERE attempts < max_attempts AND NOT cancellation_requested\n AND ((status = 'queued' AND available_at <= now())\n OR (status = 'running' AND lease_until < now()))\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED\n LIMIT 1\n )\n UPDATE horae_jobs j\n SET status = 'running', attempts = j.attempts + 1,\n lease_until = now() + $1::int * interval '1 second', worker_id = $2,\n claim_token = $3,\n started_at = COALESCE(j.started_at, now()), updated_at = now()\n FROM candidate\n WHERE j.id = candidate.id\n RETURNING j.id, j.org_id, j.payload\n" rows_affected=0 rows_returned=0 elapsed=6.264136212s elapsed_secs=6.264136212 slow_threshold=1s1982026-10-08T02:16:27.152086Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="DELETE FROM horae_job_uploads u …" db.statement="\n\nDELETE FROM horae_job_uploads u\n USING horae_jobs j\n WHERE u.job_id = j.id\n AND j.status = 'succeeded'\n AND j.finished_at < now() - interval '1 day'\n" rows_affected=0 rows_returned=0 elapsed=1.422454172s elapsed_secs=1.4224541720000001 slow_threshold=1s1992026-10-08T02:16:29.658883Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="DELETE FROM horae_jobs WHERE …" db.statement="\n\nDELETE FROM horae_jobs\n WHERE status IN ('succeeded', 'failed', 'cancelled')\n AND finished_at < now() - interval '30 days'\n" rows_affected=0 rows_returned=0 elapsed=1.107304298s elapsed_secs=1.107304298 slow_threshold=1s2002026-10-08T02:16:30.612961Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=6.566085609s elapsed_secs=6.566085609 slow_threshold=1s2012026-10-08T02:16:33.439821Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="UPDATE horae_jobs SET status …" db.statement="\n\nUPDATE horae_jobs\n SET status = CASE WHEN cancellation_requested THEN 'cancelled' ELSE 'failed' END,\n lease_until = NULL, worker_id = NULL, claim_token = NULL,\n finished_at = now(), updated_at = now(),\n last_error = CASE WHEN cancellation_requested THEN NULL\n ELSE COALESCE(last_error, 'Job attempt limit reached after interruption') END\n WHERE (attempts >= max_attempts OR cancellation_requested)\n AND (status = 'queued' OR (status = 'running' AND lease_until < now()))\n" rows_affected=0 rows_returned=0 elapsed=1.802005948s elapsed_secs=1.8020059480000001 slow_threshold=1s2022026-10-08T02:16:43.740434Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=9.360566888s elapsed_secs=9.360566888 slow_threshold=1s2032026-10-08T02:16:49.658331Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id\n FROM horae_jobs\n WHERE attempts < max_attempts AND NOT cancellation_requested\n AND ((status = 'queued' AND available_at <= now())\n OR (status = 'running' AND lease_until < now()))\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED\n LIMIT 1\n )\n UPDATE horae_jobs j\n SET status = 'running', attempts = j.attempts + 1,\n lease_until = now() + $1::int * interval '1 second', worker_id = $2,\n claim_token = $3,\n started_at = COALESCE(j.started_at, now()), updated_at = now()\n FROM candidate\n WHERE j.id = candidate.id\n RETURNING j.id, j.org_id, j.payload\n" rows_affected=0 rows_returned=0 elapsed=13.594550453s elapsed_secs=13.594550453 slow_threshold=1s2042026-10-08T02:16:56.772113Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=9.1663031s elapsed_secs=9.1663031 slow_threshold=1s2052026-10-08T02:16:59.690147Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="DELETE FROM horae_job_uploads u …" db.statement="\n\nDELETE FROM horae_job_uploads u\n USING horae_jobs j\n WHERE u.job_id = j.id\n AND j.status = 'succeeded'\n AND j.finished_at < now() - interval '1 day'\n" rows_affected=0 rows_returned=0 elapsed=4.743391306s elapsed_secs=4.743391306 slow_threshold=1s2062026-10-08T02:17:04.212322Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="DELETE FROM horae_jobs WHERE …" db.statement="\n\nDELETE FROM horae_jobs\n WHERE status IN ('succeeded', 'failed', 'cancelled')\n AND finished_at < now() - interval '30 days'\n" rows_affected=0 rows_returned=0 elapsed=1.882476167s elapsed_secs=1.882476167 slow_threshold=1s2072026-10-08T02:17:07.943264Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="UPDATE horae_jobs SET status …" db.statement="\n\nUPDATE horae_jobs\n SET status = CASE WHEN cancellation_requested THEN 'cancelled' ELSE 'failed' END,\n lease_until = NULL, worker_id = NULL, claim_token = NULL,\n finished_at = now(), updated_at = now(),\n last_error = CASE WHEN cancellation_requested THEN NULL\n ELSE COALESCE(last_error, 'Job attempt limit reached after interruption') END\n WHERE (attempts >= max_attempts OR cancellation_requested)\n AND (status = 'queued' OR (status = 'running' AND lease_until < now()))\n" rows_affected=0 rows_returned=0 elapsed=1.939104632s elapsed_secs=1.939104632 slow_threshold=1s2082026-10-08T02:17:12.677957Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=10.531491728s elapsed_secs=10.531491728 slow_threshold=1s2092026-10-08T02:17:22.277673Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id\n FROM horae_jobs\n WHERE attempts < max_attempts AND NOT cancellation_requested\n AND ((status = 'queued' AND available_at <= now())\n OR (status = 'running' AND lease_until < now()))\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED\n LIMIT 1\n )\n UPDATE horae_jobs j\n SET status = 'running', attempts = j.attempts + 1,\n lease_until = now() + $1::int * interval '1 second', worker_id = $2,\n claim_token = $3,\n started_at = COALESCE(j.started_at, now()), updated_at = now()\n FROM candidate\n WHERE j.id = candidate.id\n RETURNING j.id, j.org_id, j.payload\n" rows_affected=0 rows_returned=0 elapsed=12.413766345s elapsed_secs=12.413766344999999 slow_threshold=1s2102026-10-08T02:17:24.380313Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH due AS ( …" db.statement="\n\nWITH due AS (\n UPDATE time_entries te\n SET notified_long_running_at = now()\n FROM organizations o\n WHERE o.id = te.org_id\n AND te.is_running = true\n AND te.notified_long_running_at IS NULL\n AND te.started_at < now() - make_interval(mins => o.long_timer_minutes)\n RETURNING te.id, te.org_id, te.user_id, te.project_id, te.task_id,\n te.spent_date, te.minutes, te.billable, te.notes, te.started_at\n )\n SELECT id, org_id, user_id, project_id, task_id,\n spent_date as \"spent_date: chrono::NaiveDate\",\n minutes, billable, notes,\n started_at as \"started_at!: chrono::DateTime<chrono::Utc>\",\n (EXTRACT(EPOCH FROM (now() - started_at)) / 60)::int as \"running_minutes!\"\n FROM due\n" rows_affected=0 rows_returned=0 elapsed=37.520474462s elapsed_secs=37.520474462 slow_threshold=1s2112026-10-08T02:17:29.449586Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="DELETE FROM horae_job_uploads u …" db.statement="\n\nDELETE FROM horae_job_uploads u\n USING horae_jobs j\n WHERE u.job_id = j.id\n AND j.status = 'succeeded'\n AND j.finished_at < now() - interval '1 day'\n" rows_affected=0 rows_returned=0 elapsed=2.133469342s elapsed_secs=2.133469342 slow_threshold=1s2122026-10-08T02:17:29.485354Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=10.519745267s elapsed_secs=10.519745267 slow_threshold=1s2132026-10-08T02:17:34.772367Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="DELETE FROM horae_jobs WHERE …" db.statement="\n\nDELETE FROM horae_jobs\n WHERE status IN ('succeeded', 'failed', 'cancelled')\n AND finished_at < now() - interval '30 days'\n" rows_affected=0 rows_returned=0 elapsed=2.687217631s elapsed_secs=2.687217631 slow_threshold=1s2142026-10-08T02:17:39.446650Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="SELECT ps.project_id, ps.org_id FROM …" db.statement="\n\nSELECT ps.project_id, ps.org_id FROM project_settings ps\n JOIN projects p ON p.id = ps.project_id AND p.org_id = ps.org_id\n WHERE ps.alert_enabled AND p.active AND ps.project_id > $1\n ORDER BY ps.project_id LIMIT 100\n" rows_affected=0 rows_returned=0 elapsed=11.960335397s elapsed_secs=11.960335397 slow_threshold=1s2152026-10-08T02:17:39.446608Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="UPDATE horae_jobs SET status …" db.statement="\n\nUPDATE horae_jobs\n SET status = CASE WHEN cancellation_requested THEN 'cancelled' ELSE 'failed' END,\n lease_until = NULL, worker_id = NULL, claim_token = NULL,\n finished_at = now(), updated_at = now(),\n last_error = CASE WHEN cancellation_requested THEN NULL\n ELSE COALESCE(last_error, 'Job attempt limit reached after interruption') END\n WHERE (attempts >= max_attempts OR cancellation_requested)\n AND (status = 'queued' OR (status = 'running' AND lease_until < now()))\n" rows_affected=0 rows_returned=0 elapsed=1.811196149s elapsed_secs=1.8111961490000001 slow_threshold=1s2162026-10-08T02:17:39.601670Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=5.591269683s elapsed_secs=5.591269683 slow_threshold=1s2172026-10-08T02:17:44.274530Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=2.326089329s elapsed_secs=2.326089329 slow_threshold=1s2182026-10-08T02:17:46.670645Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH due AS ( …" db.statement="\n\nWITH due AS (\n UPDATE time_entries te\n SET notified_long_running_at = now()\n FROM organizations o\n WHERE o.id = te.org_id\n AND te.is_running = true\n AND te.notified_long_running_at IS NULL\n AND te.started_at < now() - make_interval(mins => o.long_timer_minutes)\n RETURNING te.id, te.org_id, te.user_id, te.project_id, te.task_id,\n te.spent_date, te.minutes, te.billable, te.notes, te.started_at\n )\n SELECT id, org_id, user_id, project_id, task_id,\n spent_date as \"spent_date: chrono::NaiveDate\",\n minutes, billable, notes,\n started_at as \"started_at!: chrono::DateTime<chrono::Utc>\",\n (EXTRACT(EPOCH FROM (now() - started_at)) / 60)::int as \"running_minutes!\"\n FROM due\n" rows_affected=0 rows_returned=0 elapsed=1.153678247s elapsed_secs=1.153678247 slow_threshold=1s2192026-10-08T02:17:53.480125Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id FROM horae_outbox\n WHERE delivered_at IS NULL AND failed_at IS NULL AND available_at <= now() AND event_kind = $2\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED LIMIT 1\n )\n UPDATE horae_outbox o\n SET attempts = o.attempts + 1,\n claim_token = $1,\n available_at = now() + interval '5 minutes'\n FROM candidate\n WHERE o.id = candidate.id\n RETURNING o.id, o.org_id, o.event_kind, o.payload, o.attempts,\n o.claim_token as \"claim_token!\"\n" rows_affected=0 rows_returned=0 elapsed=1.065374911s elapsed_secs=1.065374911 slow_threshold=1s2202026-10-08T02:17:54.774283Z WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="WITH candidate AS ( …" db.statement="\n\nWITH candidate AS (\n SELECT id\n FROM horae_jobs\n WHERE attempts < max_attempts AND NOT cancellation_requested\n AND ((status = 'queued' AND available_at <= now())\n OR (status = 'running' AND lease_until < now()))\n ORDER BY available_at, created_at\n FOR UPDATE SKIP LOCKED\n LIMIT 1\n )\n UPDATE horae_jobs j\n SET status = 'running', attempts = j.attempts + 1,\n lease_until = now() + $1::int * interval '1 second', worker_id = $2,\n claim_token = $3,\n started_at = COALESCE(j.started_at, now()), updated_at = now()\n FROM candidate\n WHERE j.id = candidate.id\n RETURNING j.id, j.org_id, j.payload\n" rows_affected=0 rows_returned=0 elapsed=1.894242606s elapsed_secs=1.8942426060000002 slow_threshold=1s2212026-10-08 02:06:43.932 UTC [18] LOG: starting PostgreSQL 17.10 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit2222026-10-08 02:06:43.932 UTC [18] LOG: listening on Unix socket "/tmp/horae-browser.aKxtZZ/.s.PGSQL.5432"2232026-10-08 02:06:43.934 UTC [21] LOG: database system was shut down at 2026-10-08 02:06:43 UTC2242026-10-08 02:06:43.937 UTC [18] LOG: database system is ready to accept connections2252026-10-08 02:06:45.103 UTC [47] LOG: could not send data to client: Broken pipe2262026-10-08 02:06:45.103 UTC [47] FATAL: connection to client lost2272026-10-08 02:11:44.049 UTC [19] LOG: checkpoint starting: time2282026-10-08 02:13:42.447 UTC [19] LOG: checkpoint complete: wrote 1159 buffers (7.1%); 0 WAL file(s) added, 0 removed, 0 recycled; write=118.375 s, sync=0.001 s, total=118.397 s; sync files=0, longest=0.000 s, average=0.000 s; distance=6147 kB, estimate=6147 kB; lsn=0/1B10720, redo lsn=0/1AED6D02292026-10-08 02:16:44.731 UTC [19] LOG: checkpoint starting: time2302026-10-08 02:17:30.427 UTC [19] LOG: checkpoint complete: wrote 139 buffers (0.8%); 0 WAL file(s) added, 0 removed, 0 recycled; write=43.853 s, sync=0.035 s, total=45.704 s; sync files=0, longest=0.000 s, average=0.000 s; distance=289 kB, estimate=5561 kB; lsn=0/1B35EF0, redo lsn=0/1B35E602312026-10-08 02:17:57.979 UTC [18] LOG: received fast shutdown request2322026-10-08 02:17:57.981 UTC [18] LOG: aborting any active transactions2332026-10-08 02:17:57.989 UTC [18] LOG: background worker "logical replication launcher" (PID 24) exited with exit code 12342026-10-08 02:17:57.997 UTC [19] LOG: shutting down2352026-10-08 02:17:58.021 UTC [19] LOG: checkpoint starting: shutdown immediate2362026-10-08 02:17:58.029 UTC [19] LOG: checkpoint complete: wrote 0 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.001 s, sync=0.001 s, total=0.032 s; sync files=0, longest=0.000 s, average=0.000 s; distance=0 kB, estimate=5005 kB; lsn=0/1B35FA0, redo lsn=0/1B35FA02372026-10-08 02:17:58.069 UTC [18] LOG: database system is shut down