tenderpilot/README.md

# TenderPilot MVP - Backend & Landing Page

A UK public procurement tender finder and bid assistant SaaS platform.

## Deployment Details

**Server:** 75.127.4.250 (port 22022)  
**Deployment Path:** `/home/peter/tenderpilot/`  
**Status:** ✓ Production Ready

## Quick Links

- **Landing Page:** http://75.127.4.250/
- **API Base:** http://75.127.4.250/api
- **Health Check:** http://75.127.4.250/health

## Architecture

### Frontend
- **Location:** `/public/`
- **Server:** Nginx (port 80)
- **Tech:** Vanilla HTML/CSS/JS (no build step)
- **Responsive:** Mobile-first design
- **Colors:** Navy (#1a2332), Teal (#0d9488)

### Backend API
- **Framework:** Node.js + Express
- **Port:** 3456 (behind nginx reverse proxy)
- **Database:** PostgreSQL 16
- **Auth:** JWT tokens
- **Process Manager:** PM2

### Database
- **Engine:** PostgreSQL 16
- **Name:** tenderpilot
- **User:** tenderpilot
- **Tables:**
  - `users` - User accounts and authentication
  - `tenders` - UK government tender listings
  - `profiles` - User matching profiles
  - `matches` - Tender matches for users

### Data Pipeline
- **Scraper:** Contracts Finder API (OCDS format)
- **Schedule:** Every 4 hours via cron
- **Initial Load:** 100+ tenders populated
- **Location:** `scraper.js`

## API Endpoints

### Authentication
```
POST /api/auth/register
  Body: { email, password, company_name, tier? }
  Returns: { user, token }

POST /api/auth/login
  Body: { email, password }
  Returns: { user, token }
```

### Tenders
```
GET /api/tenders?search=text&sort=deadline&limit=20&offset=0
  Headers: Authorization: Bearer <token>
  Returns: { tenders[], total }

GET /api/tenders/:id
  Headers: Authorization: Bearer <token>
  Returns: tender object
```

### User Profile & Matching
```
POST /api/profile
  Headers: Authorization: Bearer <token>
  Body: { sectors[], keywords[], min_value?, max_value?, locations[], authority_types[] }
  Returns: profile object

GET /api/matches
  Headers: Authorization: Bearer <token>
  Returns: { matches[] }
```

## File Structure

```
/home/peter/tenderpilot/
├── server.js           - Express server (3,456 lines)
├── scraper.js          - Contracts Finder scraper
├── init-db.js          - Database initialization
├── .env                - Environment configuration
├── package.json        - Dependencies
├── README.md           - This file
├── public/
│   ├── index.html      - Landing page
│   ├── styles.css      - Responsive styling
│   └── main.js         - Client-side signup form
├── node_modules/       - Dependencies
└── scraper.log         - Scraper execution log
```

## Configuration

Edit `.env` for:
- `PORT` - API server port (default: 3456)
- `DATABASE_URL` - PostgreSQL connection string
- `JWT_SECRET` - JWT signing secret (change in production!)
- `NODE_ENV` - Environment (production/development)

**Current .env:**
```
PORT=3456
DATABASE_URL=postgresql://tenderpilot:tenderpilot123@localhost:5432/tenderpilot
JWT_SECRET=your-secret-key-change-in-production-min-32-chars-long
NODE_ENV=production
```

## Running & Management

### Check Status
```bash
pm2 list                          # View all processes
pm2 logs tenderpilot-api          # View API logs
systemctl status nginx            # Check nginx
systemctl status postgresql       # Check database
```

### Manual Operations
```bash
# Start services
pm2 start server.js --name "tenderpilot-api"
sudo systemctl start nginx
sudo systemctl start postgresql

# Stop services
pm2 stop tenderpilot-api
sudo systemctl stop nginx

# Restart
pm2 restart tenderpilot-api
sudo systemctl reload nginx

# Run scraper manually
cd /home/peter/tenderpilot && node scraper.js
```

### Cron Jobs
```bash
# View scheduled jobs
crontab -l

# Current job: Scraper runs every 4 hours
0 */4 * * * cd /home/peter/tenderpilot && node scraper.js >> /home/peter/tenderpilot/scraper.log 2>&1
```

## Testing

### API Tests
```bash
# Health check
curl http://75.127.4.250/health

# Register new user
curl -X POST http://75.127.4.250/api/auth/register \
  -H "Content-Type: application/json" \
  -d '{"email":"test@example.com","password":"test123","company_name":"Test Co"}'

# Login
curl -X POST http://75.127.4.250/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email":"test@example.com","password":"test123"}'

# Get tenders (requires token)
curl -X GET "http://75.127.4.250/api/tenders?limit=5" \
  -H "Authorization: Bearer <YOUR_TOKEN>"
```

### Landing Page
- Open http://75.127.4.250 in browser
- Test signup form (beta users inserted with tier='beta')
- Check responsive design on mobile

## Monitoring & Logs

### PM2 Logs
```bash
pm2 logs tenderpilot-api --lines 100
pm2 logs tenderpilot-api --lines 50 --nostream
```

### Nginx Access/Error Logs
```bash
sudo tail -f /var/log/nginx/access.log
sudo tail -f /var/log/nginx/error.log
```

### Scraper Log
```bash
tail -f /home/peter/tenderpilot/scraper.log
```

## Persistence

### PM2 Startup
- Automatically started on server reboot
- Configuration: `/etc/systemd/system/pm2-peter.service`
- Enable: `systemctl enable pm2-peter`

### Nginx Startup
- Enabled: `sudo systemctl enable nginx`
- Starts automatically on reboot

### PostgreSQL Startup
- Enabled: `sudo systemctl enable postgresql`
- Starts automatically on reboot

## Production Considerations

1. **Security**
   - [ ] Change JWT_SECRET in .env to a strong random value
   - [ ] Enable HTTPS/SSL via Let's Encrypt
   - [ ] Rate limit API endpoints (already configured in code)
   - [ ] Add CORS whitelist for production domain

2. **Scalability**
   - [ ] Add database indexes for common queries
   - [ ] Implement caching layer (Redis)
   - [ ] Load balancing for multiple API instances

3. **Monitoring**
   - [ ] Set up error tracking (Sentry, etc.)
   - [ ] Add health check monitoring
   - [ ] Monitor scraper failures

4. **Database**
   - [ ] Implement automated backups
   - [ ] Set up replication
   - [ ] Monitor database performance

## Dependencies

Key npm packages:
- `express` - Web framework
- `pg` - PostgreSQL client
- `bcrypt` - Password hashing
- `jsonwebtoken` - JWT auth
- `cors` - Cross-origin support
- `express-rate-limit` - Rate limiting
- `axios` - HTTP client
- `dotenv` - Environment variables

## Troubleshooting

### "Permission denied" on landing page
```bash
sudo chmod 755 /home/peter
sudo chmod -R o+rX /home/peter/tenderpilot
sudo systemctl reload nginx
```

### API not responding
```bash
pm2 restart tenderpilot-api
curl http://localhost:3456/health
```

### Database connection error
```bash
sudo systemctl restart postgresql
# Check /home/peter/tenderpilot/.env DATABASE_URL
```

### Scraper not running
```bash
# Test manually
node /home/peter/tenderpilot/scraper.js
# Check cron logs
sudo journalctl -u cron
```

## Next Steps

1. Set up domain name (tenderpilot.co.uk)
2. Add SSL certificate (Let's Encrypt)
3. Implement email notifications
4. Build user dashboard
5. Add bid template library
6. Implement advanced matching algorithm
7. Add analytics dashboard

---

**Last Updated:** 2026-02-14  
**Version:** 1.0.0 (MVP)