Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KTOR-7806 Logging: Format log like OkHttp client does #4574

Closed
wants to merge 43 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
0881c4a
KTOR-7952 Fix for uninitialized writerJob property
bjhham Dec 17, 2024
f0d1cb7
KTOR-7963 Handle ClosedWatchServiceException
bjhham Dec 17, 2024
728dee2
Add Changelog for 3.0.3 (#4565)
Stexxe Dec 19, 2024
6650d28
Update dependency ch.qos.logback:logback-classic to v1.5.14 (#4563)
renovate[bot] Dec 19, 2024
2897a0d
Update dependency io.swagger.codegen.v3:swagger-codegen to v3.0.65 (#…
renovate[bot] Dec 19, 2024
72a4f5b
Update dependency io.swagger.codegen.v3:swagger-codegen-generators to…
renovate[bot] Dec 19, 2024
63a460f
Update dependency org.jetbrains.kotlinx:kotlinx-browser to v0.3 (#4457)
renovate[bot] Dec 20, 2024
08111dd
Update dependency org.jetbrains.kotlinx:kotlinx-io-core to v0.6.0 (#4…
renovate[bot] Dec 20, 2024
f351379
Update dependency org.jetbrains.kotlinx.binary-compatibility-validato…
renovate[bot] Dec 21, 2024
d083c8e
Update dependency io.swagger.codegen.v3:swagger-codegen to v3.0.66 (#…
renovate[bot] Dec 22, 2024
5d6138e
Update dependency org.freemarker:freemarker to v2.3.34 (#4572)
renovate[bot] Dec 23, 2024
b418628
Update dependency ch.qos.logback:logback-classic to v1.5.15 (#4570)
renovate[bot] Dec 23, 2024
863ee4f
KTOR-4816: UrlBuilder: support tel scheme (#4518)
anton-erofeev Dec 23, 2024
ddcd411
Update dependency io.mockk:mockk to v1.13.14 (#4569)
renovate[bot] Dec 24, 2024
8ead776
KTOR-7806 Add basic logging with standard format
Stexxe Dec 17, 2024
4ce0479
Add tests with ContentEncoding plugin
Stexxe Dec 17, 2024
c810df2
Logging for the HEADERS level
Stexxe Dec 17, 2024
e51d981
Write no logs when level is NONE
Stexxe Dec 17, 2024
5b320e2
Small refactoring
Stexxe Dec 17, 2024
fcdd253
Add tests for when streaming content includes content length
Stexxe Dec 17, 2024
3ce3d96
Log response body
Stexxe Dec 17, 2024
87032a4
Log request body
Stexxe Dec 19, 2024
64da0b2
Fix some tests
Stexxe Dec 19, 2024
7343452
Refactoring
Stexxe Dec 19, 2024
84169da
Logging chunked response body
Stexxe Dec 19, 2024
867eaa3
Fix level checks
Stexxe Dec 20, 2024
4afb061
Add Gzip response tests
Stexxe Dec 20, 2024
ca815bc
Add request for BR encoded response body
Stexxe Dec 20, 2024
27cb518
Add tests for binary response body
Stexxe Dec 20, 2024
0575014
Log connection failures
Stexxe Dec 20, 2024
8315cba
Test ALL log level
Stexxe Dec 20, 2024
897ff5d
Redact sensitive headers
Stexxe Dec 24, 2024
b6fcbd7
Add a test for receiving response body after validating the response …
Stexxe Dec 24, 2024
690eb53
Test response body size of UTF-8 body
Stexxe Dec 24, 2024
0a7f026
Fix reading a chunk from response body to detect if the body is a tex…
Stexxe Dec 24, 2024
e6835b5
Correctly log request body for all types of OutgoingContent
Stexxe Dec 25, 2024
b1d8a3f
Split request body channel on the client's scope
Stexxe Dec 25, 2024
da8c439
Add tests for Gzip-encoded request body
Stexxe Dec 25, 2024
df20b32
Fix writing buffer and avoid logging empty lines when body size is 0
Stexxe Dec 26, 2024
1b332b4
Test binary data integrity after logging
Stexxe Dec 26, 2024
887fdd6
Decrease chunk size for testing if content is binary
Stexxe Dec 26, 2024
a64af1e
Small improvements
Stexxe Dec 26, 2024
8c5cf5e
Add kdoc for the standardFormat option
Stexxe Dec 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,16 @@
# 3.0.3
> Published 18 December 2024

### Bugfixes
* "Module not found" errors when executing browserProductionWebpack task since 3.0.2 ([KTOR-7912](https://youtrack.jetbrains.com/issue/KTOR-7912))
* Darwin: "IllegalStateException: Content-Length mismatch" on requesting gzipped content ([KTOR-7943](https://youtrack.jetbrains.com/issue/KTOR-7943))
* JS/WASM fails with "IllegalStateException: Content-Length mismatch" on requesting gzipped content ([KTOR-7934](https://youtrack.jetbrains.com/issue/KTOR-7934))
* FormFieldLimit is overwritten by default arg ([KTOR-7946](https://youtrack.jetbrains.com/issue/KTOR-7946))
* A Performance issue reading with ByteReadChannel.readUTF8LineTo request body ([KTOR-7941](https://youtrack.jetbrains.com/issue/KTOR-7941))
* Installing HttpCache before ContentEncoding prevents response body to be decoded ([KTOR-7830](https://youtrack.jetbrains.com/issue/KTOR-7830))
* TestApplication.stop() doesn't stop the application anymore ([KTOR-7682](https://youtrack.jetbrains.com/issue/KTOR-7682))
* File is not commited after closing writeChannel() of the file ([KTOR-7845](https://youtrack.jetbrains.com/issue/KTOR-7845))

# 3.0.2
> Published 3 December 2024

Expand Down
16 changes: 8 additions & 8 deletions gradle/libs.versions.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@
kotlin = "2.0.21"
kotlinx-html = "0.11.0"
kotlinx-datetime = "0.6.1"
kotlinx-io = "0.5.4"
kotlinx-io = "0.6.0"
coroutines = "1.9.0"
atomicfu = "0.26.1"
serialization = "1.7.3"
binaryCompatibilityValidator = "0.16.3"
binaryCompatibilityValidator = "0.17.0"
dokka = "1.9.20"
kover = "0.9.0"
ktlint = "4.5.0"
kotlinx-browser = "0.2"
kotlinx-browser = "0.3"

# Used for test server in buildSrc
ktor = "3.1.0-eap-1190"
Expand Down Expand Up @@ -40,21 +40,21 @@ jackson = "2.18.2"

junit = "5.11.4"
slf4j = "2.0.16"
logback = "1.5.12"
logback = "1.5.15"

dropwizard = "4.2.29"
micrometer = "1.14.1"

jansi = "2.4.1"
typesafe = "1.4.3"

mockk = "1.13.13"
mockk = "1.13.14"

java-jwt = "4.4.0"

jwks-rsa = "0.22.1"
mustache = "0.9.14"
freemarker = "2.3.33"
freemarker = "2.3.34"
pebble = "3.2.2"
velocity = "2.4.1"
velocity-tools = "3.1"
Expand All @@ -70,8 +70,8 @@ ws = "8.18.0"
xmlutil = "0.90.3"
yamlkt = "0.13.0"

swagger-codegen = "3.0.64"
swagger-codegen-generators = "1.0.54"
swagger-codegen = "3.0.66"
swagger-codegen-generators = "1.0.55"
swagger-parser = "2.1.24"

puppeteer = "21.5.0"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,16 @@ internal class ByteChannelReplay(private val origin: ByteReadChannel) {
if (copyTask == null) {
copyTask = CopyFromSourceTask()
if (!content.compareAndSet(null, copyTask)) {
copyTask = content.value
// second thread, read from copy
copyTask = content.value!!
} else {
// first thread, read from origin
return copyTask.start()
}
}

return GlobalScope.writer {
val body = copyTask!!.awaitImpatiently()
val body = copyTask.awaitImpatiently()
channel.writeFully(body)
}.channel
}
Expand All @@ -44,12 +46,9 @@ internal class ByteChannelReplay(private val origin: ByteReadChannel) {
private inner class CopyFromSourceTask(
val savedResponse: CompletableDeferred<ByteArray> = CompletableDeferred()
) {
lateinit var writerJob: WriterJob
private val writerJob: WriterJob by lazy { receiveBody() }

fun start(): ByteReadChannel {
writerJob = receiveBody()
return writerJob.channel
}
fun start() = writerJob.channel

@OptIn(DelicateCoroutinesApi::class)
fun receiveBody(): WriterJob = GlobalScope.writer(Dispatchers.Unconfined) {
Expand Down
65 changes: 65 additions & 0 deletions ktor-client/ktor-client-core/common/test/ByteChannelReplayTest.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
/*
* Copyright 2014-2024 JetBrains s.r.o and contributors. Use of this source code is governed by the Apache 2.0 license.
*/
import io.ktor.client.plugins.internal.*
import io.ktor.utils.io.*
import kotlinx.coroutines.joinAll
import kotlinx.coroutines.launch
import kotlinx.coroutines.test.runTest
import kotlinx.coroutines.yield
import kotlin.test.BeforeTest
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue

internal class ByteChannelReplayTest {

private val size = 1024 * 1024 + 1
private val expectedByte = 'A'.code.toByte()
private val expected = ByteArray(size).apply { fill(expectedByte) }
private lateinit var channelReplay: ByteChannelReplay

@BeforeTest
fun setup() {
channelReplay = ByteChannelReplay(ByteReadChannel(expected))
}

@Test
fun readFirst() = runTest {
val first = channelReplay.replay()
assertRead(first)
val second = channelReplay.replay()
assertRead(second)
}

@Test
fun readSecond() = runTest {
val first = channelReplay.replay()
val second = channelReplay.replay()
assertRead(second)
assertTrue(first.isClosedForRead)
}

@Test
fun readABunch() = runTest {
val jobs = (0..10).map {
launch {
val readChannel = channelReplay.replay()
yield()
try {
assertRead(readChannel)
} catch (e: Exception) {
assertEquals("Save body abandoned", e.message)
}
}
}
joinAll(*jobs.toTypedArray())
}

private suspend fun assertRead(readChannel: ByteReadChannel) {
repeat(size) { i ->
assertEquals(expectedByte, readChannel.readByte(), "Incorrect byte at index $i")
}
assertTrue(readChannel.isClosedForRead || readChannel.exhausted())
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ kotlin.sourceSets {
jvmTest {
dependencies {
api(project(":ktor-shared:ktor-serialization:ktor-serialization-jackson"))
api(project(":ktor-client:ktor-client-plugins:ktor-client-encoding"))
}
}
}
Loading
Loading