Harden and deploy your app¶
Finally, it's time to deploy your model. But before that, you have to make sure you follow AI deployment best practices. In the past, this step was too often either the source of unexpected struggle, or worse, simply ignored.
With GreatAI, it has become a matter of 4 lines of code.
from great_ai import GreatAI, use_model
from great_ai.utilities import clean
@GreatAI.create
@use_model("my-domain-predictor")
def predict_domain(sentence, model):
inputs = [clean(sentence)]
return str(model.predict(inputs)[0])
Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️ Cannot find credentials files, defaulting to using ParallelTinyDbDriver The selected tracing database (ParallelTinyDbDriver) is not recommended for production Cannot find credentials files, defaulting to using LargeFileLocal GreatAI (v0.1.4): configured ✅ 🔩 tracing_database: ParallelTinyDbDriver 🔩 large_file_implementation: LargeFileLocal 🔩 is_production: False 🔩 should_log_exception_stack: True 🔩 prediction_cache_size: 512 🔩 dashboard_table_size: 50 You still need to check whether you follow all best practices before trusting your deployment. > Find out more at https://se-ml.github.io/practices Fetching cached versions of my-domain-predictor Latest version of my-domain-predictor is 9 (from versions: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9) File my-domain-predictor-9 found in cache
predict_domain("Mountains are just big rocks.")
Trace[str]({'created': '2022-07-12T13:34:26.743292',
'exception': None,
'feedback': None,
'logged_values': { 'arg:sentence:length': 29,
'arg:sentence:value': 'Mountains are just big rocks.'},
'models': [{'key': 'my-domain-predictor', 'version': 9}],
'original_execution_time_ms': 6.9699,
'output': 'geography',
'tags': ['predict_domain', 'online', 'development'],
'trace_id': 'c80bdee3-602b-49dd-a84d-6eef80127e5a'})
Notice how the original return value is under the .output key. Additionally, a plethora of metadata has been added which will be useful later on.
Running your app in development-mode is as easy as executing great-ai deploy.ipynb from your terminal.
!great-ai deploy.ipynb
# leave this running and open http://127.0.0.1:6060
2022-07-12 15:34:28 | INFO | Converting notebook to Python script 2022-07-12 15:34:29 | INFO | Found `predict_domain` to be the GreatAI app 2022-07-12 15:34:29 | INFO | Uvicorn running on http://0.0.0.0:6060 (Press CTRL+C to quit) 2022-07-12 15:34:31 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️ 2022-07-12 15:34:31 | WARNING | Cannot find credentials files, defaulting to using ParallelTinyDbDriver 2022-07-12 15:34:31 | WARNING | The selected tracing database (ParallelTinyDbDriver) is not recommended for production 2022-07-12 15:34:31 | WARNING | Cannot find credentials files, defaulting to using LargeFileLocal 2022-07-12 15:34:31 | INFO | GreatAI (v0.1.4): configured ✅ 2022-07-12 15:34:31 | INFO | 🔩 tracing_database: ParallelTinyDbDriver 2022-07-12 15:34:31 | INFO | 🔩 large_file_implementation: LargeFileLocal 2022-07-12 15:34:31 | INFO | 🔩 is_production: False 2022-07-12 15:34:31 | INFO | 🔩 should_log_exception_stack: True 2022-07-12 15:34:31 | INFO | 🔩 prediction_cache_size: 512 2022-07-12 15:34:31 | INFO | 🔩 dashboard_table_size: 50 2022-07-12 15:34:31 | WARNING | You still need to check whether you follow all best practices before trusting your deployment. 2022-07-12 15:34:31 | WARNING | > Find out more at https://se-ml.github.io/practices 2022-07-12 15:34:31 | INFO | Fetching cached versions of my-domain-predictor 2022-07-12 15:34:31 | INFO | Latest version of my-domain-predictor is 9 (from versions: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9) 2022-07-12 15:34:31 | INFO | File my-domain-predictor-9 found in cache 2022-07-12 15:34:31 | INFO | Started server process [199794] 2022-07-12 15:34:31 | INFO | Waiting for application startup. 2022-07-12 15:34:31 | INFO | Application startup complete. ^C 2022-07-12 15:34:33 | INFO | Shutting down 2022-07-12 15:34:33 | INFO | Waiting for application shutdown. 2022-07-12 15:34:33 | INFO | Application shutdown complete. 2022-07-12 15:34:33 | INFO | Finished server process [199794]
Congrats, you've just created your first GreatAI service! 🎉
Now that you've made sure your application is hardened enough for the intended use case, it is time to deploy it. The responsibilities of GreatAI end when it wraps your inference function and model into a production-ready service. You're given the freedom and responsibility to deploy this service. Fortunately, you (or your organisation) probably already have an established routine for deploying services.
There are three main approaches to deploy a GreatAI service: For more info about them, check out the deployment how-to.
For more thorough examples, see the examples page.