feat(v2): add audio url and predefined document#940
Conversation
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
1285a1c to
6025c2f
Compare
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
…udio-v2 Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
|
@JoanFM @alaeddine-13 I checked all the comments and made corresponding changes. It's ready for re-review now. |
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
docarray/typing/url/audio_url.py
Outdated
| ) | ||
| return cls(str(url), scheme=None) | ||
|
|
||
| def load(self: T) -> np.ndarray: |
There was a problem hiding this comment.
@samsja Should I return a np.ndarray here, or rather AudioNdArray or AudioTorchTensor?
There was a problem hiding this comment.
I would say AudioNdArray.
Normally we just load the np.ndarray, bc the framework specific conversion can happen when putting it back into the document, and having it as np.ndarray makes the least assumptions about the sorrounding code.
In this case, since there are actual audio features that come with the array, I would go with AudioNdArray. It can still be treated like a normal np.ndarray, but bring the aforementioned features.
But just verify in a test that setting a AudioNdArray to a field with type AudioTorchArray actually works without issue?
There was a problem hiding this comment.
But just verify in a test that setting a AudioNdArray to a field with type AudioTorchArray actually works without issue
Do you mean like this?
def test_load_audio_url_to_audio_torch_tensor(file_url):
class MyAudioDoc(Document):
audio_url: AudioUrl
tensor: Optional[AudioTorchTensor]
doc = MyAudioDoc(audio_url=file_url)
doc.tensor = doc.audio_url.load()
assert isinstance(doc.tensor, np.ndarray)
assert isinstance(doc.tensor, AudioNdArray)
There was a problem hiding this comment.
Ok I moved it to the computational backends
samsja
left a comment
There was a problem hiding this comment.
This pr looks really nice. I like how the idea of having different tensor types for the different modality work :) I added some comments
JohannesMessner
left a comment
There was a problem hiding this comment.
Great PR, love the audio tensor stuff, this is a good pattern to use moving forward. just some things to consider
docarray/typing/url/audio_url.py
Outdated
| ) | ||
| return cls(str(url), scheme=None) | ||
|
|
||
| def load(self: T) -> np.ndarray: |
There was a problem hiding this comment.
I would say AudioNdArray.
Normally we just load the np.ndarray, bc the framework specific conversion can happen when putting it back into the document, and having it as np.ndarray makes the least assumptions about the sorrounding code.
In this case, since there are actual audio features that come with the array, I would go with AudioNdArray. It can still be treated like a normal np.ndarray, but bring the aforementioned features.
But just verify in a test that setting a AudioNdArray to a field with type AudioTorchArray actually works without issue?
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
JohannesMessner
left a comment
There was a problem hiding this comment.
Just resolve the conflicts and we're good to go, great work!
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
|
📝 Docs are deployed on https://ft-feat-add-audio-v2--jina-docs.netlify.app 🎉 |
Add support for audio files to docarray v2
Goals:
AudioUrlAudioTensorwithAudioNdarrayandAudioTorchTensorAudiopredefined doc (with optional attrsAudio.tensor,Audio.urlandAudio.embedding)