Vibrant Color Brightness and Hue Proximity Filtering

[bscholer]

Checklist

• I have filled out the template to the best of my ability.
• This only contains 1 feature request (if you have multiple feature requests, open one feature request for each feature request).
• This issue is not a duplicate feature request of previous feature requests.

Is your feature request related to a problem? Please describe.

I learned about this library in this thread, and I'm super interested in the Vibrant Color functionality.

I did this same sort of thing with NodeRED+JS a little while ago to automatically set the colors of my accent WLED lights. It has served as a really great source of dynamic colors, and keeps things fresh.

I've spent a good bit of time tweaking this for my use-case, and one of the things I had to implement was a way to make sure the colors that I set the lights to weren't too similar. Most album art has a handful of colors, but I found that often, it would identify very similar colors as the most prominent ones, which doesn't make for a great multi-color palette.

I've played around with this integration a bit, and it seems to me like it's more meant for use-cases where you're grabbing a single color from the palette returned by get_image_vibrant_colors. Please correct me if I'm wrong, but that's the vibe I got.

Describe the solution you'd like

It would be nice if this integration had a way to get multiple prominent colors that aren't too similar, as I'm doing in my use-case. The implementation I landed on is as follows:

1. Get the top 20 prominent colors using the get-image-colors library
2. Calc brightness for each (r + g + b, 0-765), and filter out any colors not within the range of 200-600 (dark colors don't pop on the lights, and whites aren't fun)
   Note: I totally acknowledge that this is a weird way to do this, but I found that using the (L)ightness from HSL didn't do as good a job of representing the perceived brightness of these colors. I'm sure there's a better way, but I'm not super well-versed in color theory, and this implementation is a super simple, and has worked great.
3. Find hues for each in the HSL colorspace (0-360), and remove any that are within 20 of another color. This is the most important piece of this imo
4. Set the lights to a pattern using the top 3 colors left after filtering (this had to be done through the WLED API and was a headache of its own, but not within scope of this)

I've found this to do a great job of grabbing colors that look great, and stand out.

That all said, it would be great if SpotifyPlus had a way to do something similar. That said, the existing implementation of this service returns a well-defined palette, as opposed to my more arbitrary list of colors.

As such, I think the cleanest way to implement this would be to add a few more optional parameters:

• brightness min
• brightness max
• hue separation (degrees in HSL colorspace)
  If unset, these would not change the existing behavior.

The brightness range would be used to filter all the colors in the palette, and the hue separation could be used to either filter all the colors returned, or just colors within each of the 3 groups (dark, light, general).

Describe alternatives you've considered

Just doing a bit of poking around, it seems like using relative luminance or the L* component of the CIELab colorspace would be a more standard approach that may yield better results.

I'm absolutely open to feedback on how this functionality might fit in best to the existing implementation. I'd love to switch my setup over to this library to avoid my NodeRED spaghetti :)

Additional context

My existing NodeRED flow
Short Demo

[bscholer]

I'd love to help out with this implementation, but want to make sure that the feature is designed well and fits into this well before digging into it.

Also, I realize that the implementation of this existing feature is in SpotifyWebApiPython. If you'd like me to move this issue over there, I'm happy to. Just figured I'd put it here since this repo has more stars/visibility.

[thlucas1]

@bscholer
The "vibrant" namespace was imported from the vibrant-python library, as the latest vibrant-python version has a dependency that was not compatible with current version of Home Assistant (vibrant-python==0.1.6 depends on pillow>=10.1.0,<11.0.0). It also appears that vibrant-python is no longer maintained.

I have not delved too much into the vibrant code, as it's pretty intense and I am not well versed image processing. The gist of it is that it returns information for 6 of the most commonly used colors in the image. There are several attributes returned for each of the 6 colors: rgb value, rgb hex value, population, and hsl value. I believe the population is the number of times that color was found in the image.

The only input parameters to the vibrant method are:

• image_source - The image source to extract color palette information from. If the prefix of the value is http: or https:, then the image is downloaded from the url. This can also point to a filename on the local file system. If null, the currently playing Spotify track image url is used.
• color_count - The number of colors in the initial palette from which swatches will be generated. Default is 64; Range is 1 to 256.
• color_quality - Controls the processing time and quality of the palette generation. A lower value (e.g. 1) results in higher quality but takes more processing time, while a higher value (e.g. 5) is faster but may result in a lower-quality palette. Default is 5; Range is 1 to 10.

Service Call

action: spotifyplus.get_image_vibrant_colors
data:
  entity_id: media_player.spotifyplus_todd_l
  image_source: https://i.scdn.co/image/ab67616d0000b2733deaee5f76ab2da15dd8db86
  color_count: 64
  color_quality: 5

Service Response

user_profile:
  display_name: Todd L
...
result:
  dark_muted:
    rgb_hex: "#762524"
    rgb:
      - 118
      - 37
      - 36
    population: 0
    hsl:
      - 0.732
      - 0.532
      - 0.302
  dark_vibrant:
    rgb_hex: "#7d2726"
    rgb:
      - 125
      - 39
      - 38
    population: 5970
    hsl:
      - 0.69
      - 0.534
      - 0.32
  light_muted:
    rgb_hex: "#937b07"
    rgb:
      - 147
      - 123
      - 7
    population: 0
    hsl:
      - 49.87
      - 0.917
      - 0.3
  light_vibrant:
    rgb_hex: "#f8de5e"
    rgb:
      - 248
      - 222
      - 94
    population: 2711
    hsl:
      - 49.87
      - 0.917
      - 0.671
  muted:
    rgb_hex: "#937a07"
    rgb:
      - 147
      - 122
      - 7
    population: 0
    hsl:
      - 49.286
      - 0.909
      - 0.302
  vibrant:
    rgb_hex: "#f6d12a"
    rgb:
      - 246
      - 209
      - 42
    population: 47235
    hsl:
      - 49.118
      - 0.919
      - 0.565

With that said, I would probably implement a new service for your requested functionality. Do you know if there is a Python equivalent to the NodeJS get-image-colors function? I found the following on the web via a Google search:

from PIL import Image
import numpy as np
from collections import Counter

def get_image_colors(image_path, num_colors=5):
    """
    Extracts the most dominant colors from an image.

    Args:
        image_path (str): Path to the image file.
        num_colors (int): Number of dominant colors to extract.

    Returns:
        list: List of tuples representing the RGB color values.
    """
    try:
        img = Image.open(image_path).convert('RGB')
    except FileNotFoundError:
        print(f"Error: Image not found at '{image_path}'")
        return []
    
    img_array = np.array(img)
    pixels = img_array.reshape(-1, 3)

    color_counts = Counter(map(tuple, pixels))
    most_common_colors = color_counts.most_common(num_colors)
    
    return [color for color, count in most_common_colors]

# Example Usage
image_file = 'your_image.jpg'  # Replace with your image path
dominant_colors = get_image_colors(image_file, num_colors=5)

if dominant_colors:
    print("Dominant Colors (RGB):")
    for color in dominant_colors:
        print(color)

[thlucas1]

@bscholer
I finally found some time to take a look at this request.

I was able to modify an existing implementation of ColorThief, and think I have things narrowed down to what you specified.

I took the 3 images in your YouTube video and ran them through the processor. Note that I have not implemented this yet into the integration; just a proof of concept at this time. I used the same parameters for each image:

• palette color_count=20
• brightness_filter_low=200
• brightness_filter_high=600
• hue_distance_filter=20)

Below are the results.
The "Dezko, Ascend" (spotify:track:199wv1uOJYZ1XyK8FTzwh2) track only generated 1 palette entry.

Does this track with what you are expecting?
Any other Spotify album / track images I can test with?

Francis Lai Laven, La leçon particulière (spotify:track:3NWRapFZSlpHXO38LW21rw)

Dezko, Ascend (spotify:track:199wv1uOJYZ1XyK8FTzwh2)

Orbit, Summer Someday (spotify:track:2uETu1hY79BdvX9wRqQgrh)

[bscholer]

@thlucas1 awesome!! I'm about to move, and will be seeing Home Assistant up from scratch, so it'll be a great opportunity to migrate things over!

Regarding more tracks to test with, I have a playlist for this exact purpose! Only 6 tracks in there, but it's probably a decent enough test case. Hopefully that helps!
https://open.spotify.com/playlist/10zDRUNVBUnxMRkXPQTr9X?si=1WA65tZgRWq5qD7xP2Vfpg&pi=vL45YVPwR9ia3

Regarding Ascend, that looks perfect to me. With that returned, half(ish) of the LEDs would be blue, and the rest would be off, which matches up with the artwork perfectly.

Francis Lai Laven, La leçon particulière looks to be the perfect sort of artwork for this approach to shine: Two main, contrasting colors. The color selections are spot on!

Awesome work! I probably won't have much time to test it out in the next few weeks with the move, but I should be able to get around to it eventually!

[thlucas1]

@bscholer
Just released v1.0.163 that contains the new Get Image Palette Colors service; see the link for docs.

I also put together a test automation to get you started. It's basic, but will give you an idea of the capabilities.

alias: SpotifyPlus Example - Light Show using Hue
description: >-
  Creates a light show effect using currently playing Spotify artwork and Hue
  lighting.
triggers:
  - alias: Trigger when SpotifyPlus media player Track ID changes
    trigger: state
    entity_id: media_player.spotifyplus_todd_l
    attribute: media_content_id
actions:
  - delay:
      hours: 0
      minutes: 0
      seconds: 2
      milliseconds: 0
    alias: Delay to give attributes time to fully update
    enabled: false
  - variables:
      spotify_uri: "{{ state_attr('media_player.spotifyplus_todd_l', 'media_content_id') }}"
      spotify_title: "{{ state_attr('media_player.spotifyplus_todd_l', 'media_title') }}"
      spotify_album: "{{ state_attr('media_player.spotifyplus_todd_l', 'media_album_name') }}"
      spotify_artist: "{{ state_attr('media_player.spotifyplus_todd_l', 'media_artist') }}"
      spotify_artist_uri: "{{ state_attr('media_player.spotifyplus_todd_l', 'sp_artist_uri') }}"
      spotify_image_url: >-
        {{ state_attr('media_player.spotifyplus_todd_l',
        'sp_nowplaying_image_url') }}
  - alias: Get Spotify now-playing image palette information
    action: spotifyplus.get_image_palette_colors
    data:
      entity_id: media_player.spotifyplus_todd_l
      image_source: "{{ spotify_image_url }}"
      color_count: 10
      color_quality: 1
      brightness_filter_low: 100
      brightness_filter_high: 600
      hue_distance_filter: 10
    response_variable: palette_response
  - variables:
      light_cycle_speed: 2
      light_rgb_colors: |-
        [
          {{ palette_response['result']['items'][0]['rgb'] }},
          {{ palette_response['result']['items'][1]['rgb'] }},
          {{ palette_response['result']['items'][2]['rgb'] }},
          {{ palette_response['result']['items'][3]['rgb'] }},
          {{ palette_response['result']['items'][4]['rgb'] }},
          {{ palette_response['result']['items'][5]['rgb'] }}
        ]
  - repeat:
      while:
        - condition: state
          alias: SpotifyPlus media player is still playing something
          entity_id: media_player.spotifyplus_todd_l
          state: playing
        - condition: template
          alias: SpotifyPlus media player nowplaying image has not changed
          value_template: >-
            {{ spotify_image_url ==
            state_attr('media_player.spotifyplus_todd_l',
            'sp_nowplaying_image_url') }}
      sequence:
        - repeat:
            count: "{{ light_rgb_colors | count }}"
            sequence:
              - action: light.turn_on
                alias: Turn light on and set RGB color
                data:
                  entity_id: light.office
                  rgb_color: "{{ light_rgb_colors[repeat.index - 1] }}"
                  brightness_pct: 100
              - action: light.turn_on
                enabled: false
                alias: Turn light on and set RGB color with 2 second transition
                data:
                  entity_id: light.office
                  rgb_color: "{{ light_rgb_colors[repeat.index - 1] }}"
                  brightness_pct: 100
                  transition: 2
              - delay:
                  seconds: "{{ light_cycle_speed }}"
mode: restart

Hope it helps!

[bscholer]

@thlucas1 amazing, thanks so much! Excited to try this out!!

[thlucas1]

@bscholer
I now have a blueprint for the SpotifyPlus Hue Lightshow functionality. This blueprint will start a Philips Hue lightshow effect when playing Spotify content via a SpotifyPlus integration media player.

Enjoy!