arrowBatch getting panic: interface conversion: arrow.Array is *array.Int64, not *array.Timestamp

[Yifeng-Sigma]

in arrowToRecord() when decoding timestampNtzType and timestampLtzType, snowflake can send int64 array, which result in panic: interface conversion: arrow.Array is *array.Int64, not *array.Timestamp because of the following code.
for i, t := range col.(*array.Timestamp).TimestampValues() {

The below fixed things.

		case timestampNtzType:
			tb := array.NewTimestampBuilder(pool, &arrow.TimestampType{Unit: arrow.Nanosecond})
			if col.DataType().ID() == arrow.STRUCT {
				structData := col.(*array.Struct)
				epoch := structData.Field(0).(*array.Int64).Int64Values()
				fraction := structData.Field(1).(*array.Int32).Int32Values()
				for i := 0; i < int(numRows); i++ {
					if !col.IsNull(i) {
						val := time.Unix(epoch[i], int64(fraction[i]))
						tb.Append(arrow.Timestamp(val.UnixNano()))
					} else {
						tb.AppendNull()
					}
				}
			} else if col.DataType().ID() == arrow.INT64 {
				for i, t := range col.(*array.Int64).Int64Values() {
					if !col.IsNull(i) {
						val := time.Unix(0, t*int64(math.Pow10(9-int(srcColumnMeta.Scale)))).UTC()
						tb.Append(arrow.Timestamp(val.UnixNano()))
					} else {
						tb.AppendNull()
					}
				}
			} else {
				for i, t := range col.(*array.Timestamp).TimestampValues() {
					if !col.IsNull(i) {
						val := time.Unix(0, int64(t)*int64(math.Pow10(9-int(srcColumnMeta.Scale)))).UTC()
						tb.Append(arrow.Timestamp(val.UnixNano()))
					} else {
						tb.AppendNull()
					}
				}
			}
			newCol = tb.NewArray()
			defer newCol.Release()
			tb.Release()
		case timestampLtzType:
			tb := array.NewTimestampBuilder(pool, &arrow.TimestampType{Unit: arrow.Nanosecond, TimeZone: loc.String()})
			if col.DataType().ID() == arrow.STRUCT {
				structData := col.(*array.Struct)
				epoch := structData.Field(0).(*array.Int64).Int64Values()
				fraction := structData.Field(1).(*array.Int32).Int32Values()
				for i := 0; i < int(numRows); i++ {
					if !col.IsNull(i) {
						val := time.Unix(epoch[i], int64(fraction[i]))
						tb.Append(arrow.Timestamp(val.UnixNano()))
					} else {
						tb.AppendNull()
					}
				}
			} else if col.DataType().ID() == arrow.INT64 {
				for i, t := range col.(*array.Int64).Int64Values() {
					if !col.IsNull(i) {
						q := t / int64(math.Pow10(int(srcColumnMeta.Scale)))
						r := t % int64(math.Pow10(int(srcColumnMeta.Scale)))
						val := time.Unix(q, r)
						tb.Append(arrow.Timestamp(val.UnixNano()))
					} else {
						tb.AppendNull()
					}
				}
			} else {
				for i, t := range col.(*array.Timestamp).TimestampValues() {
					if !col.IsNull(i) {
						q := int64(t) / int64(math.Pow10(int(srcColumnMeta.Scale)))
						r := int64(t) % int64(math.Pow10(int(srcColumnMeta.Scale)))
						val := time.Unix(q, r)
						tb.Append(arrow.Timestamp(val.UnixNano()))
					} else {
						tb.AppendNull()
					}
				}
			}
			newCol = tb.NewArray()
			defer newCol.Release()
			tb.Release()

[sfc-gh-dszmolka]

hello and thank you for submitting this issue ! do you think you could provide a minimal viable reproduction code snippet which when run, exhibits the error you're encountering?

[Yifeng-Sigma]

if you add

{
 			logical:  "timestamp_ltz",
 			physical: "int64",
 			values:   []time.Time{time.Now(), localTime},
 			nrows:    2,
 			rowType:  execResponseRowType{Scale: 9},
 			sc:       arrow.NewSchema([]arrow.Field{{Type: &arrow.Int64Type{}}}, nil),
 			builder:  array.NewInt64Builder(pool),
 			append: func(b array.Builder, vs interface{}) {
 				for _, t := range vs.([]time.Time) {
 					b.(*array.Int64Builder).Append(t.UnixNano())
 				}
 			},
 			compare: func(src interface{}, convertedRec arrow.Record) int {
 				srcvs := src.([]time.Time)
 				for i, t := range convertedRec.Column(0).(*array.Timestamp).TimestampValues() {
 					if srcvs[i].UnixNano() != int64(t) {
 						return i
 					}
 				}
 				return -1
 			},
 		},

to TestArrowRecord() you will be able to see it. Basically server can send int64 as timestamp.

[sfc-gh-dszmolka]

thank you , this looks helpful! we'll take a look.

[sfc-gh-dszmolka]

this should also be fixed with PR #918 - thank you for bearing with us while this is being worked on !

[sfc-gh-dszmolka]

PR merged and will be part of the October release, expected towards the end of October

[sfc-gh-dszmolka]

gosnowflake 1.7.0 released with the fix, thank you all for bearing with us !