Update the default buildpacks to use paketo and add a tip for tilt

[sambhav]

Signed-off-by: Sambhav Kothari skothari44@bloomberg.net

Paketo is a sibling Linux Foundation project under the CFF umbrella. It also supports live reloading through tilt which allows for a better local dev experience.

[kserve-oss-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: samj1912
To complete the pull request process, please assign theofpa after the PR has been reviewed.
You can assign the PR to them by writing /assign @theofpa in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[netlify]

✔️ Deploy Preview for elastic-nobel-0aef7a ready!

🔨 Explore the source changes: 698d3f8

🔍 Inspect the deploy log: https://app.netlify.com/sites/elastic-nobel-0aef7a/deploys/621a41501f6e5d0008ac4e5d

😎 Browse the preview: https://deploy-preview-119--elastic-nobel-0aef7a.netlify.app

[yuzisun]

@samj1912 getting the following error with paketobuildpack, it is working fine with heroku buildpack or manually crafted Dockerfile.

Traceback (most recent call last):
  File "/layers/paketo-buildpacks_cpython/cpython/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/layers/paketo-buildpacks_cpython/cpython/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/workspace/model.py", line 15, in <module>
    import kserve
  File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.8/site-packages/kserve/__init__.py", line 17, in <module>
    from kserve.model_server import ModelServer
  File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.8/site-packages/kserve/model_server.py", line 27, in <module>
    import kserve.handlers as handlers
  File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.8/site-packages/kserve/handlers/__init__.py", line 15, in <module>
    from .base import NotFoundHandler  # noqa # pylint: disable=unused-import
  File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.8/site-packages/kserve/handlers/base.py", line 18, in <module>
    from kserve.model_repository import ModelRepository
  File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.8/site-packages/kserve/model_repository.py", line 17, in <module>
    from ray.serve.api import RayServeHandle
  File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.8/site-packages/ray/serve/__init__.py", line 2, in <module>
    from ray.serve.api import (start, get_replica_context, shutdown, ingress,
  File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.8/site-packages/ray/serve/api.py", line 25, in <module>
    from ray.serve.controller import ServeController
  File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.8/site-packages/ray/serve/controller.py", line 13, in <module>
    from ray.serve.deployment_state import ReplicaState, DeploymentStateManager
  File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.8/site-packages/ray/serve/deployment_state.py", line 19, in <module>
    from ray.serve.storage.kv_store import KVStoreBase
  File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.8/site-packages/ray/serve/storage/kv_store.py", line 1, in <module>
    import sqlite3
  File "/layers/paketo-buildpacks_cpython/cpython/lib/python3.8/sqlite3/__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "/layers/paketo-buildpacks_cpython/cpython/lib/python3.8/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: libsqlite3.so.0: cannot open shared object file: No such file or directory

[sambhav]

@yuzisun it should work now.

[yuzisun]

This looks cool, you can create the Tiltfile like following for continuous dev development, we might want to add a bit more instruction for how to do this.

load('ext://pack', 'pack')
k8s_kind('InferenceService', api_version='serving.kserve.io/v1beta1', image_json_path='{.spec.predictor.containers[*].image}')

allow_k8s_contexts('docker-desktop')

pack(
  'yuzisun/tilt-demo-2022',
  ignore=[
    'Tiltfile',
    'service.yaml',
    '.*',
  ],
  live_update = [
    sync('.', '/workspace'),
  ],
  deps=["."],
)
k8s_yaml('service.yaml')
k8s_resource('tilt-demo', extra_pod_selectors={'serving.kserve.io/inferenceservice':'tilt-demo'})

[sambhav]

The tiltfile I am using looks a bit like -

# -*- mode: Python -*-

version_settings(constraint='>=0.22.1')
load('ext://pack', 'pack')
docker_compose('docker-compose.yml')

pack('myapp',
  builder="paketobuildpacks/builder:full",
  live_update=[
    sync('.', '/workspace'),
  ]
)