ISSUE/346: 支持CublasLT，实现了LT的linear，fp8 linear，fp8 block-wise linear，和fp8 的 group-wise quant #510
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
有些代码需要做一些格式的format |
qinyiqun
force-pushed
the
fp8
branch
2 times, most recently
from
abb321d to
104c424
Compare
…cuBLASLt and fp8 group-wise quant
qinyiqun
requested review from
PanZezhong1725 and
Ziminli
and removed request for
PanZezhong1725
PanZezhong1725
requested changes
Oct 21, 2025
| INFINI_STATUS_BAD_TENSOR_SHAPE = 11, | ||
| INFINI_STATUS_BAD_TENSOR_STRIDES = 12, | ||
| INFINI_STATUS_INSUFFICIENT_WORKSPACE = 13, | ||
| INFINI_STATUS_NOT_ALIGNED = 14, |
There was a problem hiding this comment.
需要在InfiniCore/src/utils/infini_status_string.h中添加string
| INFINI_DTYPE_C64 = 17, | ||
| INFINI_DTYPE_C128 = 18, | ||
| INFINI_DTYPE_BF16 = 19, | ||
| INFINI_DTYPE_F8_E4M3 = 20, |
There was a problem hiding this comment.
这个会影响python前端接口，和 @voltjia 过一下
There was a problem hiding this comment.
这个没关系，我在 infinicore Python 层还会再封装一次，所以只要能跟 torch 里的对上号就行。
| __C __export infiniStatus_t | ||
| infiniopDestroyLinearDescriptor(infiniopLinearDescriptor_t desc); | ||
|
|
||
| #endif No newline at end of file |
| from numpy.lib.stride_tricks import as_strided | ||
|
|
||
| from .. import InfiniopTestWriter, InfiniopTestCase, np_dtype_to_ggml, gguf_strides, contiguous_gguf_strides, process_zero_stride_tensor | ||
| from .. import ( |
| @@ -0,0 +1,406 @@ | |||
| import torch | |||
There was a problem hiding this comment.
如果测试都是分成两个，那是不是分成两个算子会更好一点？
|
|
||
| typedef struct InfiniopDescriptor *infiniopQuantizeDescriptor_t; | ||
|
|
||
| __C __export infiniStatus_t infiniopCreateQuantizeDescriptor( |
There was a problem hiding this comment.
这不是通用的quantize，在命名的时候应该注明是什么quantize
| #include <utility> | ||
|
|
||
| /** | ||
| * @brief Define the process for initializing a Descriptor of an elementwise operation |
|
写一下接口设计文档，有的参数的含义需要解释 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A100 只支持F16及以上精度
H100 支持FP8E4M3和Fp8E5M2
FP8-Block-Wise
FP8
其他
FP8-Quantize-Group-Wise