Initial commit: TenderPilot MVP
This commit is contained in:
Peter Foster
295
README.md
Normal file
295
README.md
Normal file
@@ -0,0 +1,295 @@
|
||||
# TenderPilot MVP - Backend & Landing Page
|
||||
|
||||
A UK public procurement tender finder and bid assistant SaaS platform.
|
||||
|
||||
## Deployment Details
|
||||
|
||||
**Server:** 75.127.4.250 (port 22022)
|
||||
**Deployment Path:** `/home/peter/tenderpilot/`
|
||||
**Status:** ✓ Production Ready
|
||||
|
||||
## Quick Links
|
||||
|
||||
- **Landing Page:** http://75.127.4.250/
|
||||
- **API Base:** http://75.127.4.250/api
|
||||
- **Health Check:** http://75.127.4.250/health
|
||||
|
||||
## Architecture
|
||||
|
||||
### Frontend
|
||||
- **Location:** `/public/`
|
||||
- **Server:** Nginx (port 80)
|
||||
- **Tech:** Vanilla HTML/CSS/JS (no build step)
|
||||
- **Responsive:** Mobile-first design
|
||||
- **Colors:** Navy (#1a2332), Teal (#0d9488)
|
||||
|
||||
### Backend API
|
||||
- **Framework:** Node.js + Express
|
||||
- **Port:** 3456 (behind nginx reverse proxy)
|
||||
- **Database:** PostgreSQL 16
|
||||
- **Auth:** JWT tokens
|
||||
- **Process Manager:** PM2
|
||||
|
||||
### Database
|
||||
- **Engine:** PostgreSQL 16
|
||||
- **Name:** tenderpilot
|
||||
- **User:** tenderpilot
|
||||
- **Tables:**
|
||||
- `users` - User accounts and authentication
|
||||
- `tenders` - UK government tender listings
|
||||
- `profiles` - User matching profiles
|
||||
- `matches` - Tender matches for users
|
||||
|
||||
### Data Pipeline
|
||||
- **Scraper:** Contracts Finder API (OCDS format)
|
||||
- **Schedule:** Every 4 hours via cron
|
||||
- **Initial Load:** 100+ tenders populated
|
||||
- **Location:** `scraper.js`
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Authentication
|
||||
```
|
||||
POST /api/auth/register
|
||||
Body: { email, password, company_name, tier? }
|
||||
Returns: { user, token }
|
||||
|
||||
POST /api/auth/login
|
||||
Body: { email, password }
|
||||
Returns: { user, token }
|
||||
```
|
||||
|
||||
### Tenders
|
||||
```
|
||||
GET /api/tenders?search=text&sort=deadline&limit=20&offset=0
|
||||
Headers: Authorization: Bearer <token>
|
||||
Returns: { tenders[], total }
|
||||
|
||||
GET /api/tenders/:id
|
||||
Headers: Authorization: Bearer <token>
|
||||
Returns: tender object
|
||||
```
|
||||
|
||||
### User Profile & Matching
|
||||
```
|
||||
POST /api/profile
|
||||
Headers: Authorization: Bearer <token>
|
||||
Body: { sectors[], keywords[], min_value?, max_value?, locations[], authority_types[] }
|
||||
Returns: profile object
|
||||
|
||||
GET /api/matches
|
||||
Headers: Authorization: Bearer <token>
|
||||
Returns: { matches[] }
|
||||
```
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
/home/peter/tenderpilot/
|
||||
├── server.js - Express server (3,456 lines)
|
||||
├── scraper.js - Contracts Finder scraper
|
||||
├── init-db.js - Database initialization
|
||||
├── .env - Environment configuration
|
||||
├── package.json - Dependencies
|
||||
├── README.md - This file
|
||||
├── public/
|
||||
│ ├── index.html - Landing page
|
||||
│ ├── styles.css - Responsive styling
|
||||
│ └── main.js - Client-side signup form
|
||||
├── node_modules/ - Dependencies
|
||||
└── scraper.log - Scraper execution log
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Edit `.env` for:
|
||||
- `PORT` - API server port (default: 3456)
|
||||
- `DATABASE_URL` - PostgreSQL connection string
|
||||
- `JWT_SECRET` - JWT signing secret (change in production!)
|
||||
- `NODE_ENV` - Environment (production/development)
|
||||
|
||||
**Current .env:**
|
||||
```
|
||||
PORT=3456
|
||||
DATABASE_URL=postgresql://tenderpilot:tenderpilot123@localhost:5432/tenderpilot
|
||||
JWT_SECRET=your-secret-key-change-in-production-min-32-chars-long
|
||||
NODE_ENV=production
|
||||
```
|
||||
|
||||
## Running & Management
|
||||
|
||||
### Check Status
|
||||
```bash
|
||||
pm2 list # View all processes
|
||||
pm2 logs tenderpilot-api # View API logs
|
||||
systemctl status nginx # Check nginx
|
||||
systemctl status postgresql # Check database
|
||||
```
|
||||
|
||||
### Manual Operations
|
||||
```bash
|
||||
# Start services
|
||||
pm2 start server.js --name "tenderpilot-api"
|
||||
sudo systemctl start nginx
|
||||
sudo systemctl start postgresql
|
||||
|
||||
# Stop services
|
||||
pm2 stop tenderpilot-api
|
||||
sudo systemctl stop nginx
|
||||
|
||||
# Restart
|
||||
pm2 restart tenderpilot-api
|
||||
sudo systemctl reload nginx
|
||||
|
||||
# Run scraper manually
|
||||
cd /home/peter/tenderpilot && node scraper.js
|
||||
```
|
||||
|
||||
### Cron Jobs
|
||||
```bash
|
||||
# View scheduled jobs
|
||||
crontab -l
|
||||
|
||||
# Current job: Scraper runs every 4 hours
|
||||
0 */4 * * * cd /home/peter/tenderpilot && node scraper.js >> /home/peter/tenderpilot/scraper.log 2>&1
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### API Tests
|
||||
```bash
|
||||
# Health check
|
||||
curl http://75.127.4.250/health
|
||||
|
||||
# Register new user
|
||||
curl -X POST http://75.127.4.250/api/auth/register \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"email":"test@example.com","password":"test123","company_name":"Test Co"}'
|
||||
|
||||
# Login
|
||||
curl -X POST http://75.127.4.250/api/auth/login \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"email":"test@example.com","password":"test123"}'
|
||||
|
||||
# Get tenders (requires token)
|
||||
curl -X GET "http://75.127.4.250/api/tenders?limit=5" \
|
||||
-H "Authorization: Bearer <YOUR_TOKEN>"
|
||||
```
|
||||
|
||||
### Landing Page
|
||||
- Open http://75.127.4.250 in browser
|
||||
- Test signup form (beta users inserted with tier='beta')
|
||||
- Check responsive design on mobile
|
||||
|
||||
## Monitoring & Logs
|
||||
|
||||
### PM2 Logs
|
||||
```bash
|
||||
pm2 logs tenderpilot-api --lines 100
|
||||
pm2 logs tenderpilot-api --lines 50 --nostream
|
||||
```
|
||||
|
||||
### Nginx Access/Error Logs
|
||||
```bash
|
||||
sudo tail -f /var/log/nginx/access.log
|
||||
sudo tail -f /var/log/nginx/error.log
|
||||
```
|
||||
|
||||
### Scraper Log
|
||||
```bash
|
||||
tail -f /home/peter/tenderpilot/scraper.log
|
||||
```
|
||||
|
||||
## Persistence
|
||||
|
||||
### PM2 Startup
|
||||
- Automatically started on server reboot
|
||||
- Configuration: `/etc/systemd/system/pm2-peter.service`
|
||||
- Enable: `systemctl enable pm2-peter`
|
||||
|
||||
### Nginx Startup
|
||||
- Enabled: `sudo systemctl enable nginx`
|
||||
- Starts automatically on reboot
|
||||
|
||||
### PostgreSQL Startup
|
||||
- Enabled: `sudo systemctl enable postgresql`
|
||||
- Starts automatically on reboot
|
||||
|
||||
## Production Considerations
|
||||
|
||||
1. **Security**
|
||||
- [ ] Change JWT_SECRET in .env to a strong random value
|
||||
- [ ] Enable HTTPS/SSL via Let's Encrypt
|
||||
- [ ] Rate limit API endpoints (already configured in code)
|
||||
- [ ] Add CORS whitelist for production domain
|
||||
|
||||
2. **Scalability**
|
||||
- [ ] Add database indexes for common queries
|
||||
- [ ] Implement caching layer (Redis)
|
||||
- [ ] Load balancing for multiple API instances
|
||||
|
||||
3. **Monitoring**
|
||||
- [ ] Set up error tracking (Sentry, etc.)
|
||||
- [ ] Add health check monitoring
|
||||
- [ ] Monitor scraper failures
|
||||
|
||||
4. **Database**
|
||||
- [ ] Implement automated backups
|
||||
- [ ] Set up replication
|
||||
- [ ] Monitor database performance
|
||||
|
||||
## Dependencies
|
||||
|
||||
Key npm packages:
|
||||
- `express` - Web framework
|
||||
- `pg` - PostgreSQL client
|
||||
- `bcrypt` - Password hashing
|
||||
- `jsonwebtoken` - JWT auth
|
||||
- `cors` - Cross-origin support
|
||||
- `express-rate-limit` - Rate limiting
|
||||
- `axios` - HTTP client
|
||||
- `dotenv` - Environment variables
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Permission denied" on landing page
|
||||
```bash
|
||||
sudo chmod 755 /home/peter
|
||||
sudo chmod -R o+rX /home/peter/tenderpilot
|
||||
sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
### API not responding
|
||||
```bash
|
||||
pm2 restart tenderpilot-api
|
||||
curl http://localhost:3456/health
|
||||
```
|
||||
|
||||
### Database connection error
|
||||
```bash
|
||||
sudo systemctl restart postgresql
|
||||
# Check /home/peter/tenderpilot/.env DATABASE_URL
|
||||
```
|
||||
|
||||
### Scraper not running
|
||||
```bash
|
||||
# Test manually
|
||||
node /home/peter/tenderpilot/scraper.js
|
||||
# Check cron logs
|
||||
sudo journalctl -u cron
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Set up domain name (tenderpilot.co.uk)
|
||||
2. Add SSL certificate (Let's Encrypt)
|
||||
3. Implement email notifications
|
||||
4. Build user dashboard
|
||||
5. Add bid template library
|
||||
6. Implement advanced matching algorithm
|
||||
7. Add analytics dashboard
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2026-02-14
|
||||
**Version:** 1.0.0 (MVP)
|
||||
Reference in New Issue
Block a user