Engine: PostgreSQL 16
Name: tenderpilot
User: tenderpilot
Tables:
- users - User accounts and authentication
- tenders - UK government tender listings
- profiles - User matching profiles
- matches - Tender matches for users

Data Pipeline

Scraper: Contracts Finder API (OCDS format)
Schedule: Every 4 hours via cron
Initial Load: 100+ tenders populated
Location: scraper.js

API Endpoints

Authentication

POST /api/auth/register
  Body: { email, password, company_name, tier? }
  Returns: { user, token }

POST /api/auth/login
  Body: { email, password }
  Returns: { user, token }

Tenders

GET /api/tenders?search=text&sort=deadline&limit=20&offset=0
  Headers: Authorization: Bearer <token>
  Returns: { tenders[], total }

GET /api/tenders/:id
  Headers: Authorization: Bearer <token>
  Returns: tender object

User Profile & Matching

POST /api/profile
  Headers: Authorization: Bearer <token>
  Body: { sectors[], keywords[], min_value?, max_value?, locations[], authority_types[] }
  Returns: profile object

GET /api/matches
  Headers: Authorization: Bearer <token>
  Returns: { matches[] }

File Structure

/home/peter/tenderpilot/
├── server.js           - Express server (3,456 lines)
├── scraper.js          - Contracts Finder scraper
├── init-db.js          - Database initialization
├── .env                - Environment configuration
├── package.json        - Dependencies
├── README.md           - This file
├── public/
│   ├── index.html      - Landing page
│   ├── styles.css      - Responsive styling
│   └── main.js         - Client-side signup form
├── node_modules/       - Dependencies
└── scraper.log         - Scraper execution log

Configuration

Edit .env for:

PORT - API server port (default: 3456)
DATABASE_URL - PostgreSQL connection string
JWT_SECRET - JWT signing secret (change in production!)
NODE_ENV - Environment (production/development)

Current .env:

PORT=3456
DATABASE_URL=postgresql://tenderpilot:tenderpilot123@localhost:5432/tenderpilot
JWT_SECRET=your-secret-key-change-in-production-min-32-chars-long
NODE_ENV=production

Running & Management

Check Status

pm2 list                          # View all processes
pm2 logs tenderradar-api          # View API logs
systemctl status nginx            # Check nginx
systemctl status postgresql       # Check database

Manual Operations

# Start services
pm2 start server.js --name "tenderradar-api"
sudo systemctl start nginx
sudo systemctl start postgresql

# Stop services
pm2 stop tenderradar-api
sudo systemctl stop nginx

# Restart
pm2 restart tenderradar-api
sudo systemctl reload nginx

# Run scraper manually
cd /home/peter/tenderpilot && node scraper.js

Cron Jobs

# View scheduled jobs
crontab -l

# Current job: Scraper runs every 4 hours
0 */4 * * * cd /home/peter/tenderpilot && node scraper.js >> /home/peter/tenderpilot/scraper.log 2>&1

Testing

API Tests

# Health check
curl http://75.127.4.250/health

# Register new user
curl -X POST http://75.127.4.250/api/auth/register \
  -H "Content-Type: application/json" \
  -d '{"email":"test@example.com","password":"test123","company_name":"Test Co"}'

# Login
curl -X POST http://75.127.4.250/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email":"test@example.com","password":"test123"}'

# Get tenders (requires token)
curl -X GET "http://75.127.4.250/api/tenders?limit=5" \
  -H "Authorization: Bearer <YOUR_TOKEN>"

Landing Page

Open http://75.127.4.250 in browser
Test signup form (beta users inserted with tier='beta')
Check responsive design on mobile

Monitoring & Logs

PM2 Logs

pm2 logs tenderradar-api --lines 100
pm2 logs tenderradar-api --lines 50 --nostream

Nginx Access/Error Logs

sudo tail -f /var/log/nginx/access.log
sudo tail -f /var/log/nginx/error.log

Scraper Log

tail -f /home/peter/tenderpilot/scraper.log

Persistence

PM2 Startup

Automatically started on server reboot
Configuration: /etc/systemd/system/pm2-peter.service
Enable: systemctl enable pm2-peter

Nginx Startup

Enabled: sudo systemctl enable nginx
Starts automatically on reboot

PostgreSQL Startup

Enabled: sudo systemctl enable postgresql
Starts automatically on reboot

Production Considerations

Security
- Change JWT_SECRET in .env to a strong random value
- Enable HTTPS/SSL via Let's Encrypt
- Rate limit API endpoints (already configured in code)
- Add CORS whitelist for production domain
Scalability
- Add database indexes for common queries
- Implement caching layer (Redis)
- Load balancing for multiple API instances
Monitoring
- Set up error tracking (Sentry, etc.)
- Add health check monitoring
- Monitor scraper failures
Database
- Implement automated backups
- Set up replication
- Monitor database performance

Dependencies

Key npm packages:

express - Web framework
pg - PostgreSQL client
bcrypt - Password hashing
jsonwebtoken - JWT auth
cors - Cross-origin support
express-rate-limit - Rate limiting
axios - HTTP client
dotenv - Environment variables

Troubleshooting

"Permission denied" on landing page

sudo chmod 755 /home/peter
sudo chmod -R o+rX /home/peter/tenderpilot
sudo systemctl reload nginx

API not responding

pm2 restart tenderradar-api
curl http://localhost:3456/health

Database connection error

sudo systemctl restart postgresql
# Check /home/peter/tenderpilot/.env DATABASE_URL

Scraper not running

# Test manually
node /home/peter/tenderpilot/scraper.js
# Check cron logs
sudo journalctl -u cron

Next Steps

Set up domain name (tenderradar.co.uk)
Add SSL certificate (Let's Encrypt)
Implement email notifications
Build user dashboard
Add bid template library
Implement advanced matching algorithm
Add analytics dashboard

Last Updated: 2026-02-14
Version: 1.0.0 (MVP)