Reviewing Code and Spec Compliance with Skills

We got a PR #3859 about adding SIP to pipecat.ai recently and I did a quick review basing it solely on memory from my past implementations of the RTP spec. Working in a field for 20 years means we have rewritten these protocols a few times and some things are top of mind. However, LLMs are reasoning engines and they can comb through both the specifications and the code side-by-side very quickly. What they can struggle with is hidden assumptions and gotchas. Those come from experience.

On a whim, I wrote up two skills: /rfc-finder and /spec-compliance:

/rfc-finder is a discovery tool. Given a topic, protocol name, or even a code snippet like sendNack(), it searches IETF Datatracker and the RFC Editor to find the related specifications. It traces draft-to-RFC lineages (important because IETF drafts get renamed when they graduate), checks obsolescence chains (so you do not cite a spec that was superseded a decade ago, e.g., DTMF/RFC4733 instead of RFC2833), and ranks results by how foundational each one is. Under the hood it runs WebSearch against datatracker.ietf.org and rfc-editor.org, then uses WebFetch to verify metadata — status, "Obsoleted by" relationships, section numbers. It returns annotated links, not summaries: the spec itself is the source of truth.
/spec-compliance is an auditing tool. Given a code file and a specific spec section (e.g., RFC 3550 Section 5.1), it fetches that section, extracts every normative statement — the RFC 2119 keywords (MUST, SHOULD, MAY, in all-caps) — then reads the code and classifies each requirement as Met, Missing, Partial, or N/A with line-number evidence. Under the hood it uses WebFetch to pull the spec section, Read to load the source, and Grep to search adjacent modules and tests before marking something as missing. The output is a structured compliance report with a summary table.

The two skills are designed to chain: /rfc-finder answers "what specs apply here?" and /spec-compliance answers "does the code actually follow them?"

A good test of testing out these skills was PR #3859, which adds a FreeSWITCH SIP/RTP transport to Pipecat. It is a protocol implementation PR, which means every file maps onto a specific IETF specification. /rfc-finder looked at the files and inferred that the PR implements a minimal SIP UAS transport (RFC 3261 ) with G.711 codecs, RTP packetization, SDP offer/answer, and RFC 2833 DTMF. Here are the relevant RFCs found by /rfc-finder:

File	Protocol area	Primary spec
`codecs.py`	G.711 μ-law encode/decode	ITU-T G.711 (LUT tables), RFC 3551 §4.5.14 (PCMU PT)
`rtp.py`	RTP header packing, 20ms send loop	RFC 3550 §5.1 and §5.3, no §6
`rtp.py`	DTMF detection	RFC 4733 §2.3 (PT) even though the code said RFC 2833
`sdp.py`	SDP generation and parsing, offer/answer	RFC 8866 (fields), RFC 3264 §5 (Offer) and §6 (Answer)
`signaling.py`	SIP message parsing, request/response building	§13 (INVITE dialog), §15 (BYE), §17 (Transactions), no REGISTER, no CANCEL. RFC 6337 (SIP O/A)

This is the ideal shape for spec-compliance analysis: the code is explicitly implementing wire protocols and every function has a normative "should behave like X" defined somewhere in a standards document. The question is whether it actually is a better review tool than a generic /review tools provided by the coding agents.

What follows is the findings from the /rfc-finder analysing all the files listed above. I test ran this on rtp.py and shared the feedback, the developer acknowledged to fix the issue. This was a useful iteration!

isometric view of sf

Thanks for running the spec compliance check @vr000m! Pushed a commit addressing the three RFC 3550 §5.1 partial-compliance items:

Unknown payload types (MUST) — _handle_packet() now ignores packets with PTs other than PCMU (0) and DTMF (101), instead of blindly decoding as G.711.

SSRC collision detection (MUST) — If an incoming packet carries our own SSRC, we regenerate. Minimal implementation suitable for 1:1 SIP calls.

New SSRC on address change (SHOULD) — start() now regenerates the SSRC when the remote transport address changes.

I eventually ran this on the whole PR and adding constraints that it is not a universal SIP implementation, the developer has scoped it to only supporting FreeSWITCH and that the SIP servers and pipecat run on the same subnet, i.e., RTP and SIP connections are within the local network. Hence there is no RTCP, no ICE/STUN, no SIP REGISTER. With that constraint in mind, the LLM instead of re-flagging the deliberate engineering choices that were already known and discussed, reviewed subsections of the RFC that would still be relevant and important.

File	Requirement	RFC	Status	Notes
rtp.py	Marker bit handling	RFC 3550 §5.1	Not implemented	Marker bit is always 0. RFC 3551 §4.1 says the marker bit SHOULD be set on the first packet after silence suppression. Acceptable for continuous audio.
rtp.py	CSRC handling	RFC 3550 §5.1	N/A	CC=0, no mixers — correct for point-to-point
rtp.py	RTCP	RFC 3550 §6	Not implemented	Explicitly documented as out-of-scope (LAN-only). This is a known deviation. RFC 3550 says RTCP "SHOULD" be used. For LAN-only deployments with FreeSWITCH this is pragmatically fine.
rtp.py	Dynamic PT negotiation via SDP	RFC 4733 §5	Hardcoded	PT=101 is hardcoded, not negotiated from SDP `a=rtpmap:101 telephone-event/8000`. Works with FreeSWITCH defaults but technically should be negotiated.
sdp.py	SDP answer matches offer codecs	RFC 3264 §6.1	Simplified	Answer always offers PCMU regardless of what's in the offer. If the offerer doesn't support PCMU, the call will fail. Acceptable for FreeSWITCH (always supports PCMU).
signaling.py	Branch parameter in Via (UAS BYE)	RFC 3261 §8.1.1.7	Simplified	Uses `z9hG4bK{call_id[:8]}` — the magic cookie is correct but the uniqueness comes from truncated call-id. Should be fine in practice but not cryptographically unique per spec.
signaling.py	Header parsing (case sensitivity)	RFC 3261 §7.3.1	Case-sensitive	The parser does exact-case matching for header names. RFC 3261 says header names are case-insensitive. E.g., `call-id:` would not match `Call-ID`. FreeSWITCH uses standard casing, so this works in practice.

Unless we had tests or a veteran implementer's eye, the above would have been found with rigorous integration and scenario testing. This took 5 minutes to generate and an experienced reviewer can still triage and whittle down the list to an actionable implementation plan. Next step is integrating this into the PR review workflow as a github action so it can be run on protocol-touching PRs.