I'm trying to deploy a CNN model created using Tensorflow to Heroku with FastAPI. The app runs on Heroku but returns an error when trying to make model predictions. Running heroku logs --tail returns this:
2022-02-17T03:32:12.426547 00:00 app[web.1]: [2022-02-17 03:32:12 0000] [10] [ERROR] Exception in ASGI application
2022-02-17T03:32:12.426549 00:00 app[web.1]: Traceback (most recent call last):
2022-02-17T03:32:12.426549 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 373, in run_asgi
2022-02-17T03:32:12.426550 00:00 app[web.1]: result = await app(self.scope, self.receive, self.send)
2022-02-17T03:32:12.426550 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
2022-02-17T03:32:12.426551 00:00 app[web.1]: return await self.app(scope, receive, send)
2022-02-17T03:32:12.426551 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/fastapi/applications.py", line 212, in __call__
2022-02-17T03:32:12.426552 00:00 app[web.1]: await super().__call__(scope, receive, send)
2022-02-17T03:32:12.426552 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/starlette/applications.py", line 112, in __call__
2022-02-17T03:32:12.426552 00:00 app[web.1]: await self.middleware_stack(scope, receive, send)
2022-02-17T03:32:12.426553 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/starlette/middleware/errors.py", line 181, in __call__
2022-02-17T03:32:12.426553 00:00 app[web.1]: raise exc
2022-02-17T03:32:12.426554 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/starlette/middleware/errors.py", line 159, in __call__
2022-02-17T03:32:12.426554 00:00 app[web.1]: await self.app(scope, receive, _send)
2022-02-17T03:32:12.426554 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/starlette/exceptions.py", line 82, in __call__
2022-02-17T03:32:12.426554 00:00 app[web.1]: raise exc
2022-02-17T03:32:12.426555 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/starlette/exceptions.py", line 71, in __call__
2022-02-17T03:32:12.426555 00:00 app[web.1]: await self.app(scope, receive, sender)
2022-02-17T03:32:12.426555 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/starlette/routing.py", line 656, in __call__
2022-02-17T03:32:12.426555 00:00 app[web.1]: await route.handle(scope, receive, send)
2022-02-17T03:32:12.426556 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/starlette/routing.py", line 259, in handle
2022-02-17T03:32:12.426556 00:00 app[web.1]: await self.app(scope, receive, send)
2022-02-17T03:32:12.426556 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/starlette/routing.py", line 61, in app
2022-02-17T03:32:12.426556 00:00 app[web.1]: response = await func(request)
2022-02-17T03:32:12.426557 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/fastapi/routing.py", line 250, in app
2022-02-17T03:32:12.426557 00:00 app[web.1]: response = actual_response_class(response_data, **response_args)
2022-02-17T03:32:12.426557 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/starlette/responses.py", line 49, in __init__
2022-02-17T03:32:12.426558 00:00 app[web.1]: self.body = self.render(content)
2022-02-17T03:32:12.426558 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/starlette/responses.py", line 174, in render
2022-02-17T03:32:12.426558 00:00 app[web.1]: return json.dumps(
2022-02-17T03:32:12.426559 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/json/__init__.py", line 234, in dumps
2022-02-17T03:32:12.426559 00:00 app[web.1]: return cls(
2022-02-17T03:32:12.426559 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/json/encoder.py", line 199, in encode
2022-02-17T03:32:12.426560 00:00 app[web.1]: chunks = self.iterencode(o, _one_shot=True)
2022-02-17T03:32:12.426560 00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/json/encoder.py", line 257, in iterencode
2022-02-17T03:32:12.426560 00:00 app[web.1]: return _iterencode(o, 0)
2022-02-17T03:32:12.426561 00:00 app[web.1]: ValueError: Out of range float values are not JSON compliant
Most notably on the last line, it says `ValueError: Out of range float values are not JSON compliant". My procfile looks like this:
web: gunicorn -w 2 -k uvicorn.workers.UvicornWorker main:app
When trying to run the server on WSL using python -m uvicorn main:app, I get this error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/eruaro/.local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 375, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "/home/eruaro/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
return await self.app(scope, receive, send)
File "/home/eruaro/.local/lib/python3.8/site-packages/fastapi/applications.py", line 212, in __call__
await super().__call__(scope, receive, send)
File "/home/eruaro/.local/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/eruaro/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
raise exc
File "/home/eruaro/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
await self.app(scope, receive, _send)
File "/home/eruaro/.local/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
raise exc
File "/home/eruaro/.local/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
await self.app(scope, receive, sender)
File "/home/eruaro/.local/lib/python3.8/site-packages/starlette/routing.py", line 656, in __call__
await route.handle(scope, receive, send)
File "/home/eruaro/.local/lib/python3.8/site-packages/starlette/routing.py", line 259, in handle
await self.app(scope, receive, send)
File "/home/eruaro/.local/lib/python3.8/site-packages/starlette/routing.py", line 61, in app
response = await func(request)
File "/home/eruaro/.local/lib/python3.8/site-packages/fastapi/routing.py", line 250, in app
response = actual_response_class(response_data, **response_args)
File "/home/eruaro/.local/lib/python3.8/site-packages/starlette/responses.py", line 49, in __init__
self.body = self.render(content)
File "/home/eruaro/.local/lib/python3.8/site-packages/starlette/responses.py", line 174, in render
return json.dumps(
File "/usr/lib/python3.8/json/__init__.py", line 234, in dumps
return cls(
File "/usr/lib/python3.8/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python3.8/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
ValueError: Out of range float values are not JSON compliant
It's similar to the one I get on Heroku. However, on Windows, using the same command like the one I used on WSL, the app works. No error is returned and I'm able to make predictions on the server. How do I remove the JSON compliant error?
For reference, my code is in a singular main.py file:
from fastapi import FastAPI
from tensorflow.keras.models import load_model
from tensorflow.keras.utils import get_file
from tensorflow.keras.utils import load_img
from tensorflow.keras.utils import img_to_array
from tensorflow import expand_dims
from tensorflow.nn import softmax
from numpy import argmax
from numpy import max
from numpy import array
app = FastAPI()
model_dir = "food-vision-model.h5"
model = load_model(model_dir)
class_predictions = array([
'apple_pie',
'baby_back_ribs',
'baklava',
'beef_carpaccio',
'beef_tartare',
'beet_salad',
'beignets',
'bibimbap',
'bread_pudding',
'breakfast_burrito',
'bruschetta',
'caesar_salad',
'cannoli',
'caprese_salad',
'carrot_cake',
'ceviche',
'cheesecake',
'cheese_plate',
'chicken_curry',
'chicken_quesadilla',
'chicken_wings',
'chocolate_cake',
'chocolate_mousse',
'churros',
'clam_chowder',
'club_sandwich',
'crab_cakes',
'creme_brulee',
'croque_madame',
'cup_cakes',
'deviled_eggs',
'donuts',
'dumplings',
'edamame',
'eggs_benedict',
'escargots',
'falafel',
'filet_mignon',
'fish_and_chips',
'foie_gras',
'french_fries',
'french_onion_soup',
'french_toast',
'fried_calamari',
'fried_rice',
'frozen_yogurt',
'garlic_bread',
'gnocchi',
'greek_salad',
'grilled_cheese_sandwich',
'grilled_salmon',
'guacamole',
'gyoza',
'hamburger',
'hot_and_sour_soup',
'hot_dog',
'huevos_rancheros',
'hummus',
'ice_cream',
'lasagna',
'lobster_bisque',
'lobster_roll_sandwich',
'macaroni_and_cheese',
'macarons',
'miso_soup',
'mussels',
'nachos',
'omelette',
'onion_rings',
'oysters',
'pad_thai',
'paella',
'pancakes',
'panna_cotta',
'peking_duck',
'pho',
'pizza',
'pork_chop',
'poutine',
'prime_rib',
'pulled_pork_sandwich',
'ramen',
'ravioli',
'red_velvet_cake',
'risotto',
'samosa',
'sashimi',
'scallops',
'seaweed_salad',
'shrimp_and_grits',
'spaghetti_bolognese',
'spaghetti_carbonara',
'spring_rolls',
'steak',
'strawberry_shortcake',
'sushi',
'tacos',
'takoyaki',
'tiramisu',
'tuna_tartare',
'waffles'
])
@app.get("/")
async def root():
return {"message": "Welcome to the Food Vision API!"}
@app.post("/net/image/prediction/")
async def get_net_image_prediction(image_link: str = ""):
if image_link == "":
return {"message": "No image link provided"}
img_path = get_file(
origin = image_link
)
img = load_img(
img_path,
target_size = (224, 224)
)
img_array = img_to_array(img)
img_array = expand_dims(img_array, 0)
pred = model.predict(img_array)
score = softmax(pred[0])
class_prediction = class_predictions[argmax(score)]
model_score = round(max(score) * 100, 2)
return {
"model_prediction_class": class_prediction,
"model_prediction_score": model_score
}
CodePudding user response：
This error usually occurs when JSON attempts to convert a NaN value and fails to do so. Probably, tensorflow returns NaN at some point. I would advise you to try debugging by looking for a NaN value
CodePudding user response：
I've solved the issue, the solution is to dockerize the whole application and then deploy it to Heroku. That way it can work when running on WSL (Linux), and by extension on Heroku as well (which uses Linux).
This thus requires editing the main.py file by a little bit:
from fastapi import FastAPI
from tensorflow.keras.models import load_model
from tensorflow.keras.utils import get_file
from tensorflow.keras.utils import load_img
from tensorflow.keras.utils import img_to_array
from tensorflow import expand_dims
from tensorflow.nn import softmax
from numpy import argmax
from numpy import max
from numpy import array
from json import dumps
from uvicorn import run
import os
app = FastAPI()
model_dir = "food-vision-model.h5"
model = load_model(model_dir)
class_predictions = array([
'apple_pie',
'baby_back_ribs',
'baklava',
'beef_carpaccio',
'beef_tartare',
'beet_salad',
'beignets',
'bibimbap',
'bread_pudding',
'breakfast_burrito',
'bruschetta',
'caesar_salad',
'cannoli',
'caprese_salad',
'carrot_cake',
'ceviche',
'cheesecake',
'cheese_plate',
'chicken_curry',
'chicken_quesadilla',
'chicken_wings',
'chocolate_cake',
'chocolate_mousse',
'churros',
'clam_chowder',
'club_sandwich',
'crab_cakes',
'creme_brulee',
'croque_madame',
'cup_cakes',
'deviled_eggs',
'donuts',
'dumplings',
'edamame',
'eggs_benedict',
'escargots',
'falafel',
'filet_mignon',
'fish_and_chips',
'foie_gras',
'french_fries',
'french_onion_soup',
'french_toast',
'fried_calamari',
'fried_rice',
'frozen_yogurt',
'garlic_bread',
'gnocchi',
'greek_salad',
'grilled_cheese_sandwich',
'grilled_salmon',
'guacamole',
'gyoza',
'hamburger',
'hot_and_sour_soup',
'hot_dog',
'huevos_rancheros',
'hummus',
'ice_cream',
'lasagna',
'lobster_bisque',
'lobster_roll_sandwich',
'macaroni_and_cheese',
'macarons',
'miso_soup',
'mussels',
'nachos',
'omelette',
'onion_rings',
'oysters',
'pad_thai',
'paella',
'pancakes',
'panna_cotta',
'peking_duck',
'pho',
'pizza',
'pork_chop',
'poutine',
'prime_rib',
'pulled_pork_sandwich',
'ramen',
'ravioli',
'red_velvet_cake',
'risotto',
'samosa',
'sashimi',
'scallops',
'seaweed_salad',
'shrimp_and_grits',
'spaghetti_bolognese',
'spaghetti_carbonara',
'spring_rolls',
'steak',
'strawberry_shortcake',
'sushi',
'tacos',
'takoyaki',
'tiramisu',
'tuna_tartare',
'waffles'
])
@app.get("/")
async def root():
return {"message": "Welcome to the Food Vision API!"}
@app.post("/net/image/prediction/")
async def get_net_image_prediction(image_link: str = ""):
if image_link == "":
return {"message": "No image link provided"}
img_path = get_file(
origin = image_link
)
img = load_img(
img_path,
target_size = (224, 224)
)
img_array = img_to_array(img)
img_array = expand_dims(img_array, 0)
pred = model.predict(img_array)
score = softmax(pred[0])
class_prediction = class_predictions[argmax(score)]
model_score = round(max(score) * 100, 2)
model_score = dumps(model_score.tolist())
return {
"model_prediction_class": class_prediction,
"model_prediction_score": model_score
}
if __name__ == "__main__":
port = int(os.environ.get('PORT', 5000))
run(app, host="0.0.0.0", port=port)
The main difference here is the use of the os.environ.get(). I found that just manually defining a port like port = 5000 returns an R10 (boot timeout) error on Heroku.
From here, I created a Dockerfile with the following contents:
FROM python:3.7.3-stretch
# Maintainer info
LABEL maintainer="[email protected]"
# Make working directories
RUN mkdir -p /my-directory/project
WORKDIR /my-directory/project
# Upgrade pip with no cache
RUN pip install --no-cache-dir -U pip
# Copy application requirements file to the created working directory
COPY requirements.txt .
# Install application dependencies from the requirements file
RUN pip install -r requirements.txt
# Copy every file in the source folder to the created working directory
COPY . .
# Run the python application
CMD ["python", "main.py"]
With requirements.txt having these:
fastapi==0.73.0
gunicorn==20.1.0
numpy==1.19.5
uvicorn==0.15.0
image==1.5.33
tensorflow-cpu==2.7.0
I used tensorflow-cpu to bypass the slug size and memory limit of Heroku on a free account.
From here I was then able to build the docker image and deploy it to heroku.
$ docker image build -t app-name .
$ heroku create app-name
$ heroku container:push web --app app-name
$ heroku container:release web --app food-vision-api
When running locally, I used the command below:
$ docker run -p 5000:5000 -d app-name