python - ISSUES Defining Cron jobs in Procfile (Heroku) using apscheduler for Django project -
hi stackoverflow community, having problem scheduling cron job requires scraping website , storing part of model (movie) in database. problem model seems loaded before procfile executed. how should create cron job runs internally in background , storing scraped information database? here codes:
procfile:
web: python manage.py runserver 0.0.0.0:$port scheduler: python cinemas/scheduler.py
scheduler.py:
# more code above cinemas.models import movie apscheduler.schedulers.blocking import blockingscheduler sched = blockingscheduler() @sched.scheduled_job('cron', day_of_week='mon-fri', hour=0, minutes=26) def get_movies_playing_now(): global url_movies_playing_now movie.objects.all().delete() while(url_movies_playing_now): title = [] description = [] #create beatifulsoup object url link s = requests.get(url_movies_playing_now, headers=headers) soup = bs4.beautifulsoup(s.text, "html.parser") movies = soup.find_all('ul', class_='w462')[0] #find movie's title movie_title in movies.find_all('h3'): title.append(movie_title.text) #find movie's description movie_description in soup.find_all('ul', class_='w462')[0].find_all('p'): description.append(movie_description.text.replace(" [more]",".")) t, d in zip(title, description): m = movie(movie_title=t, movie_description=d) m.save() #go next page find more movies paging = soup.find( class_='pagenating').find_all('a', class_=lambda x: x != "inactive") href = "" p in paging: if "next" in p.text.lower(): href = p['href'] url_movies_playing_now = href sched.start() # more code below django.db import models
cinemas/models.py:
#create models here. class movie(models.model): movie_title = models.charfield(max_length=200) movie_description = models.charfield(max_length=20200)
this error getting when job ran.
2016-11-17t17:57:06.074914+00:00 app[scheduler.1]: traceback (most recent call last): 2016-11-17t17:57:06.074931+00:00 app[scheduler.1]: file "cinemas/scheduler.py", line 2, in 2016-11-17t17:57:06.075058+00:00 app[scheduler.1]: import cineplex 2016-11-17t17:57:06.075060+00:00 app[scheduler.1]: file "/app/cinemas/cineplex.py", line 1, in 2016-11-17t17:57:06.075173+00:00 app[scheduler.1]: cinemas.models import movie 2016-11-17t17:57:06.075196+00:00 app[scheduler.1]: file "/app/cinemas/models.py", line 5, in 2016-11-17t17:57:06.075295+00:00 app[scheduler.1]: class movie(models.model): 2016-11-17t17:57:06.075297+00:00 app[scheduler.1]: file "/app/.heroku/python/lib/python3.5/site-packages/django/db/models/base.py", line 105, in new 2016-11-17t17:57:06.075414+00:00 app[scheduler.1]: app_config = apps.get_containing_app_config(module) 2016-11-17t17:57:06.075440+00:00 app[scheduler.1]: file "/app/.heroku/python/lib/python3.5/site-packages/django/apps/registry.py", line 237, in get_containing_app_config 2016-11-17t17:57:06.075585+00:00 app[scheduler.1]: self.check_apps_ready() 2016-11-17t17:57:06.075586+00:00 app[scheduler.1]: file "/app/.heroku/python/lib/python3.5/site-packages/django/apps/registry.py", line 124, in check_apps_ready 2016-11-17t17:57:06.075703+00:00 app[scheduler.1]: raise appregistrynotready("apps aren't loaded yet.") 2016-11-17t17:57:06.075726+00:00 app[scheduler.1]: django.core.exceptions.appregistrynotready: apps aren't loaded yet.
cron job works fine if not include model objects. how should run job every day using model objects without failing?
thanks
that's because can't import django packages, models, etc.; work properly, django internals require initialization that's triggered manage.py
. rather try , re-create myself, write long-running, non-web commands custom management command (see https://docs.djangoproject.com/en/1.10/howto/custom-management-commands/). example, if app cinemas
, would:
- create
./cinemas/management/commands/scheduler.py
. - in file, sub-class
django.core.management.base.basecommand
(that sub-class must calledcommand
) - in class, override
handle()
. in case, that's you'd callsched.start()
- your
procfile
havescheduler: python manage.py scheduler
hope helps.
Comments
Post a Comment