A ACG crawler for crawling the ACG news!
A ACG crawler for crawling the ACG news!
- subscribe page
- unscribe page
- store subscribed email lists
- get subscribed email lists
- send email via MailGun API
- send log email via MailGun API
(enhancement) crawl more resources
finish the Deployment section
- (security) add CSRF-token for every pages
(Including AnimeNews, AnimeHotNews, unscribe/subscribe email address)
- (security) user input validation (email)
We assume that we have installed the JSP environment in our target host.
sudo apt-get install gradle
)gradle tomcatRunWar
to generate the acg-crawler.war
.i(The war file is in the /path/to/acg-crawler/build/libs)Auth.ini
to set the Mailgun info and GMAIL info.acg-crawler.jar
.assets
folder to the WEB-INF folder in WAR file.auth.ini
in the same directory path.java -jar /path/to/acg-crawler.jar
to crawl data, send email and send error log mail.[MAILGUN]
api-key=key-XXXXXXXXXXX
domain-name=peter279k.com.tw
api-base-url=https://api.mailgun.net/v3/peter279k.com.tw/messages
[email protected]
from-email-account=AnimeNews <[email protected]>
[GMAIL]
account=your-gmail-addresss
password=your-gmail-password
#!/bin/bash
echo "This project has built in the Ubuntu 16.04LTS (in development environment)"
sudo apt-get install gradle git-core
sudo apt-get install default-jdk default-jre tomcat7
git clone https://github.com/peter279k/acg-crawler.git
cd acg-crawler/
cp -r ./assets src/main/webapp/WEB-INF
# remember to generate the runnable jar file from eclipse IDE.
gradle clean
gradle war
sudo mkdir /home/tomcat7
sudo chown -R tomcat7 /home/tomcat7
sudo chmod u+wrx /home/tomcat7 -R
sudo cp build/libs/acg-crawler.war /var/lib/tomcat7/webapps
sudo service tomcat7 restart
echo "Finish the deployment and visit the url: domain-name:8080/acg-crawler"
echo "\n"
echo "Don't forget to use scp uploading your jar and war files"
If you found some vulnerabilities about this web application project, please feel free to send the email to [email protected].
Thanks!