673196961. Push events only. Covers all repos, current and future.How a push to a DC Bar repository reaches the cluster in seconds instead of minutes — and what to check when it stops.
673196961. Push events only. Covers all repos, current and future.https://prod-argocd.dcbar.org/api/webhookargocd-secret key webhook.github.secret. Must match the GitHub webhook's Secret field exactly./api/webhook with a Bypass policy limited to GitHub's hook IP ranges. The host-wide ArgoCD app and its Allow_Access policy are unchanged.repocreds-dcbar-web, scoped to the URL prefix https://github.com/DC-Bar-web. Backed by GitHub App Argo CD Image Puller (ID 4729778), installed org-wide.hello-world-tim-production and hello-world-tim-production-aws, both tracking DC-Bar-web/helloworld.git at main, path k8s/overlays/production, auto-sync enabled.timeout.reconciliation in argocd-cm. Default 180s. Still catches anything the webhook drops.Argo CD resolves credentials by exact repo URL first, then falls back to templates matched on URL prefix. One template covering the org means every new DC-Bar-web repo authenticates with no further setup — and no private key has to be handled again.
Each hop is a place the chain can break, and each breaks with a different signature. Knowing which hop failed is most of the diagnosis.
X-Hub-Signature-256, sent from GitHub's published hook IP ranges.webhook.github.secret. Mismatch returns 403.spec.source.repoURL and targetRevision. No match is not an error — Argo CD still answers 200.Start at GitHub, because it records the answer. Open the webhook, then Recent Deliveries, and expand a delivery. You want a 200, an X-Hub-Signature-256 header, and a payload whose repository.full_name is the repo you expect.
kubectl -n argocd logs -l app.kubernetes.io/name=argocd-server \
--since=45m --prefix --tail=-1 | grep -iE 'webhook|Requested app'kubectl logs deploy/argocd-server reads a single pod and will silently miss a webhook handled by the other one. Always select by label, as above. A webhook-triggered refresh logs the same Requested app '<name>' refresh line as a manual refresh, so match on timestamp when telling them apart.
| Signature | Cause | Fix |
|---|---|---|
| 403 | The secret in GitHub does not match webhook.github.secret. |
Re-patch the secret, restart argocd-server, and paste the same value into the webhook's Secret field. Never mid-edit one side only. |
| 302 or HTML body | Cloudflare Access is challenging the request — the Bypass policy is not matching the source address. | Compare the policy's IP ranges against api.github.com/meta. GitHub adds ranges without notice. |
| Timeout | The path-scoped Access app is misconfigured, or the origin is unreachable. | Confirm the app's destination is exactly prod-argocd.dcbar.org/api/webhook and that it sits above the host-wide app. |
| 200, no refresh | The payload's repo or branch matches no application. Normal for the org's other repos. Also the silent failure after a repo rename or transfer. | Compare repository.full_name and ref in the payload against each app's repoURL and targetRevision. |
| Repository not found | On an app patch: no credential covers that URL. Argo CD validates access before accepting a spec change. | Check the repo-creds template's URL prefix, and that the GitHub App's installation includes the repo. |
| Empty log grep | Usually you read one of two pods, or the push was to an unrelated repo. | Re-run the label-selected command above and cross-check GitHub's delivery list. |
prod-argocd.dcbar.org resolves to 10.20.100.237 from prod-k8s-master-01. Split-horizon DNS means requests from that host reach the ingress directly and never traverse Cloudflare, so a successful curl from there proves nothing about the Access policy. The tell is the response headers:
curl -sI https://prod-argocd.dcbar.org/ | grep -iE 'cf-ray|^server'No cf-ray means the request did not go through Cloudflare. Only an external caller — GitHub itself, in practice — exercises the real path.
GitHub redirects git operations after a transfer, so applications keep fetching and stay Healthy. But webhook matching is done against the URL configured on the app, not the redirect target. Pushes announce the new name, match nothing, and refreshes quietly fall back to polling. After any transfer or rename, repoint every app's repoURL and the credential entry covering it.
curl -s https://api.github.com/meta | jq -r '.hooks[]'Diff the output against the Bypass policy's IP ranges. Recorded at setup, 2026-09-01:
With webhooks delivering, polling is only a backstop and can be stretched to cut load on the repo server.
kubectl -n argocd patch configmap argocd-cm --type merge \
-p '{"data":{"timeout.reconciliation":"600s"}}'
kubectl -n argocd rollout restart \
statefulset/argocd-application-controller deploy/argocd-repo-serverCloudflare auto-named the path-scoped application prod-argocd, which is easily confused with the host-wide ArgoCD application beside it in the same list. Renaming it to something like ArgoCD GitHub Webhook would reduce the chance of someone editing or deleting the wrong one.
The setup is designed so that most new repos need nothing at all.
repocreds-dcbar-web template — no per-repo entry, no private key.repoURL set to the canonical https://github.com/DC-Bar-web/<name>.git, including the .git suffix, so webhook payloads match.