Flink UDF函数(Function)

接口Function -》接口FilterFunction、MapFunction、FlatMapFunction、ReduceFunction、CoMapFunction
自定义Function,实现以上函数,并重写内部的方法。

ds.reduce(new myFunction());

class myFunction implements ReduceFunction<SensorReading>() {
            @Override
            public SensorReading reduce(SensorReading value1, SensorReading value2) throws Exception {
                return new SensorReading(value1.getId(), value2.getTimestamp(), Math.max(value1.getTemperature(),value2.getTemperature()));
            }
        })

或者写成匿名类

        DataStream result = keyedStream.reduce(new ReduceFunction<SensorReading>() {
            @Override
            public SensorReading reduce(SensorReading value1, SensorReading value2) throws Exception {
                return new SensorReading(value1.getId(), value2.getTimestamp(), Math.max(value1.getTemperature(),value2.getTemperature()));
            }
        });
  1. lambda函数
 DataStream result = keyedStream.reduce((v1, v2) -> v1+v2 );

4,Rich Function
RichMapFunction
RichFlapMapFunction
RichFilterFunction

与常规函数不同的是,可以获取运行环境上下文,并通过生命周期方法实现更复杂功能。
生命周期方法:

  • open(), 算子如map执行前会被调用
  • close(), 清理工作
  • getRuntimeContext() ,运行时上下文,如并行度、
 class MyFunction extends RichMapFunction<SensorReading, Tuple2<String, Integer>> {
        @Override
        public void setRuntimeContext(RuntimeContext t) {
            super.setRuntimeContext(t);
        }

        @Override
        public RuntimeContext getRuntimeContext() {
            return super.getRuntimeContext();
        }

        @Override
        public IterationRuntimeContext getIterationRuntimeContext() {
            return super.getIterationRuntimeContext();
        }

        @Override
        public void open(Configuration parameters) throws Exception {
            super.open(parameters);
        }

        @Override
        public void close() throws Exception {
            super.close();
        }

        @Override
        public Tuple2<String, Integer> map(SensorReading value) throws Exception {
        value
            return null;
        }
    }

版权声明:本文为forerunner123原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接和本声明。