Dev Skills

Step 3 — Creating a short link

Validate the URL you are given, generate a code, and store it.

Topic 61 of 98 · 10 min to read · 50 min to build
After this topic you can
  • Write a random short-code generator using crypto/rand.
  • Validate an incoming URL and reject bad input with a 400.
  • Build the create endpoint through handler, service and store.

Every command for this step

terminal — run these in order
touch internal/service/code.go internal/service/shortener.go internal/transport/web/handler.go
go build ./... && go run ./cmd/api
# in a second terminal — the happy path
curl -s -X POST localhost:8080/links -d '{"url":"https://go.dev/doc/"}' | jq
# and the failures
curl -s -X POST localhost:8080/links -d '{"url":""}'
curl -s -X POST localhost:8080/links -d '{"url":"hello"}'
curl -s -X POST localhost:8080/links -d '{"url":"javascript:alert(1)"}'
curl -s -X POST localhost:8080/links -d 'not json'

What this step has to get right

This is the first endpoint that takes input from a stranger. It has three separate jobs: check the URL is a usable web address, make a short code that is not already taken, and store the result.

File 1 — internal/service/code.go

A short code is the part after the slash. It has to be short, safe in a URL, and hard to guess in order. Random characters from a fixed alphabet give you all three.

internal/service/code.go — create this file
package service
import (
"crypto/rand"
"math/big"
)
// No 0/O or 1/l/I — they are easy to misread when somebody types a code by hand.
const alphabet = "abcdefghijkmnopqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ23456789"
// NewCode returns a random code of n characters.
func NewCode(n int) (string, error) {
b := make([]byte, n)
max := big.NewInt(int64(len(alphabet)))
for i := range b {
k, err := rand.Int(rand.Reader, max) // crypto/rand, not math/rand
if err != nil {
return "", err
}
b[i] = alphabet[k.Int64()]
}
return string(b), nil
}
crypto/rand matters even here. math/rand is not designed to be unpredictable, so somebody could work out the codes you will hand out next and open links before their owners share them.

File 2 — internal/service/shortener.go

The service holds the rules. It validates the input, generates the code, retries if that code was taken, and asks the store to save. It knows nothing about HTTP, which is what lets you test it without a server.

internal/service/shortener.go — create this file
package service
import (
"context"
"errors"
"fmt"
"time"
"github.com/you/shortener/internal/model"
)
// Store is declared HERE, by the layer that uses it — not by the store package.
// It is exactly as big as this service needs, so a test fake has two methods.
type Store interface {
// ctx comes first: if the request is cancelled, the query stops too.
Save(ctx context.Context, l *model.Link) error
ByCode(ctx context.Context, code string) (*model.Link, error)
}
type Service struct{ store Store }
func New(s Store) *Service { return &Service{store: s} }
func (s *Service) Shorten(ctx context.Context, raw string) (*model.Link, error) {
long, err := validateURL(raw)
if err != nil {
return nil, err
}
// Codes are random, so a clash is possible. Try a few times rather than
// failing the request on bad luck.
for attempt := 0; attempt < 5; attempt++ {
code, err := NewCode(6)
if err != nil {
return nil, err
}
l := &model.Link{Code: code, LongURL: long, CreatedAt: time.Now()}
err = s.store.Save(ctx, l)
if err == nil {
return l, nil
}
if !errors.Is(err, model.ErrCodeTaken) {
return nil, err // a real failure, not a clash
}
}
return nil, fmt.Errorf("could not generate a free code")
}

File 3 — the validation

Validation deserves its own function because it is where you decide what you will accept from the internet. Parsing is not enough: url.Parse accepts plenty of strings that are not usable web addresses, including hello and javascript:alert(1).

internal/service/shortener.go — add this below
func validateURL(raw string) (string, error) {
raw = strings.TrimSpace(raw)
if raw == "" {
return "", fmt.Errorf("%w: url is required", ErrValidation)
}
if len(raw) > 2048 {
return "", fmt.Errorf("%w: url is too long", ErrValidation)
}
u, err := url.Parse(raw)
if err != nil {
return "", fmt.Errorf("%w: url is not valid", ErrValidation)
}
// Parse accepts "hello" as a valid relative URL, so check the parts we need.
if u.Scheme != "http" && u.Scheme != "https" {
return "", fmt.Errorf("%w: url must start with http:// or https://", ErrValidation)
}
if u.Host == "" {
return "", fmt.Errorf("%w: url has no host", ErrValidation)
}
return u.String(), nil
}
Restricting the scheme to http and https is a security check, not tidiness. Without it somebody can store a javascript: or file: URL, and your short link hands it to whoever clicks.

File 4 — the handler

internal/transport/web/handler.go — create this file
package web
import (
"encoding/json"
"errors"
"net/http"
"github.com/you/shortener/internal/model"
"github.com/you/shortener/internal/service"
)
type Handler struct{ svc *service.Service }
func NewHandler(s *service.Service) *Handler { return &Handler{svc: s} }
func (h *Handler) Routes(mux *http.ServeMux) {
mux.HandleFunc("POST /links", h.create)
}
func (h *Handler) create(w http.ResponseWriter, r *http.Request) {
var in struct {
URL string `json:"url"`
}
// Cap the body before reading it, or a huge request can take the server down.
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<20))
if err := dec.Decode(&in); err != nil {
Error(w, 400, "invalid json") // no package name: respond.go is package web too
return // always return after writing an error
}
l, err := h.svc.Shorten(r.Context(), in.URL)
if err != nil {
if errors.Is(err, model.ErrValidation) {
Error(w, 422, err.Error())
return
}
Error(w, 500, "could not shorten that url")
return
}
JSON(w, 201, l)
}

Wire it into main

cmd/api/main.go — add these lines inside run()
st := store.NewMemory()
svc := service.New(st)
h := web.NewHandler(svc)
mux := http.NewServeMux()
h.Routes(mux) // built from the inside out: store, service, handler

Try it, including the bad cases

terminal
curl -s -X POST localhost:8080/links -d '{"url":"https://go.dev/doc/"}'
# {"code":"kR7mQp","long_url":"https://go.dev/doc/", ...}
curl -s -X POST localhost:8080/links -d '{"url":"hello"}'
# {"error":"invalid input: url must start with http:// or https://"}
curl -s -X POST localhost:8080/links -d 'not json'
# {"error":"invalid json"}
Test the failures, not only the happy path. Bugs hide in the branches you never ran.
The short version
  • The service holds the rules and knows nothing about HTTP. That is what makes it testable.
  • Use crypto/rand for anything a stranger should not be able to guess.
  • url.Parse succeeding does not mean the URL is usable. Check the scheme and the host.
  • Cap the request body before decoding, and always return after writing an error.
Remember: Every value coming from a client is untrusted: its size, its shape and its contents. Decide what you accept in one place, and reject everything else.
Build this before moving on
Build all four files and get a short code back from curl. Then try five bad inputs — empty, not-a-url, a javascript: URL, a 3000-character URL, and malformed JSON — and check each gives a sensible status.
Then try these in your own editor
  1. 1Let the client ask for a custom code, and return 409 when it is already taken.
  2. 2Write a table-driven test for validateURL with eight cases, half of them invalid.
  3. 3Work out how many 6-character codes exist with this alphabet, then decide whether 6 is enough.
  4. 4Change the retry loop to give up after one attempt and work out how you would notice the problem in production.
Check yourselfTake the Project 1 — URL shortener test