优化内容压缩

* 取消用户设置的压缩级别,现在压缩级别通过系统自动设置
* Pool中的对象命中100万次时自动销毁,避免内存泄漏
* 降低Pool中的对象数量,避免占用太多内存
* 根据系统CPU线程数自动计算压缩级别,避免消耗太多CPU
* zstd限制解码的最大Window
* zstd使用低内存模式
This commit is contained in:
刘祥超
2024-04-16 11:32:38 +08:00
parent ae82310b92
commit 140d3bbf59
21 changed files with 125 additions and 82 deletions

View File

@@ -6,6 +6,8 @@ import (
"io"
)
const maxReadHits = 1 << 20
type ReaderPool struct {
c chan Reader
newFunc func(reader io.Reader) (Reader, error)
@@ -49,6 +51,11 @@ func (this *ReaderPool) Get(parentReader io.Reader) (Reader, error) {
}
func (this *ReaderPool) Put(reader Reader) {
if reader.IncreaseHit() > maxReadHits {
// do nothing to discard it
return
}
select {
case this.c <- reader:
default: