Skip to main content

Joint Real-Time Scheduling Methods for Deterministic Industrial Field/Backhaul Networks
draft-wang-detnet-joint-scheduling-01

The information below is for an old version of the document.
Document Type
This is an older version of an Internet-Draft whose latest revision state is "Expired".
Authors Heng Wang , Ping Wang , Hang Yang
Last updated 2017-05-11
RFC stream (None)
Formats
Additional resources
Stream Stream state (No stream defined)
Consensus boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date (None)
Responsible AD (None)
Send notices to (None)
draft-wang-detnet-joint-scheduling-01
DetNet                                                         H. Wang
Internet Draft                                                 P. Wang
Intended status: Standards Track                               H. Yang
Expires: November 12, 2017                     Chongqing University of
                                          Posts and Telecommunications
                                                          May 11, 2017

      Joint Real-Time Scheduling Methods for Deterministic Industrial
                          Field/Backhaul Networks
                   draft-wang-detnet-joint-scheduling-01

Abstract

   In industrial field/backhaul networks, the joint real-time
   scheduling method is important to keep end-to-end data streams
   meeting the deadline. This document proposes four joint scheduling
   methods, and the four methods consider time slotting the industrial
   backhaul network, regarding industrial backhaul network as a black
   box system, ignoring delay of industrial backhaul and establishing
   latency model of an industrial backhaul network.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other documents
   at any time. It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

   This Internet-Draft will expire on November 12, 2017.

Wang, et al.          Expires November 12, 2017               [Page 1]
 Internet-Draft           DetNet Joint Scheduling              May 2017

Copyright Notice

   Copyright (c) 2017 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with
   respect to this document. Code Components extracted from this
   document must include Simplified BSD License text as described in
   Section 4.e of the Trust Legal Provisions and are provided without
   warranty as described in the Simplified BSD License.

Table of Contents

   1. Introduction ................................................. 2
   2. Deterministic Industrial Field-Backhaul Network Requirement .. 4
   3. Deterministic Industrial Field-Backhaul Network Joint Scheduling
   Key Technology .................................................. 5
      3.1. End-to-end Network Data Stream .......................... 5
      3.2. Network Communication Resource .......................... 5
      3.3. Network Time Slot Scheduling ............................ 6
   4. Joint Real-Time Scheduling Methods for Deterministic Industrial
   Field-Backhaul Network .......................................... 6
      4.1. Time-Slotted Industrial Backhaul Networks ............... 7
      4.2. Consider Industrial Backhaul Network as a Black Box .... 10
      4.3. Ignore the Delay of Industrial Backhaul Network ........ 11
      4.4. Build Delay Model of Industrial Backhaul Network ....... 11
   5. Security Considerations ..................................... 11
   6. IANA Considerations ......................................... 11
   7. References .................................................. 11
      7.1. Normative References ................................... 11
      7.2. Informative References ................................. 11
   Authors' Addresses ............................................. 13

1. Introduction

   Industrial field network is a network that can be deployed in
   industrial process and monitor industrial field equipment and
   systems to achieve the target of control and management. It can
   improve production efficiency, reduce human intervention to
   industrial production process and decrease the cost of production.
   It has significant importance for industrial modernization.

Wang, et al.          Expires November 12, 2017               [Page 2]
 Internet-Draft           DetNet Joint Scheduling              May 2017

   Industrial field bus and industrial ethernet are two kinds of common
   solutions to industrial automation with the development of
   industrial field network, however they are both wired network. If
   they can combine the technology of wireless sensor network, a new
   network, industrial wireless network, can free from being bonded to
   wires and cables, and is more easy and flexible to deployment.
   Industrial wireless network is a communication network which is
   oriented toward building automation, and process automation, and
   industrial automation. There are three major international standards
   (ISA100.11a[IEC62734], WirelessHART[IEC62591], WIA-PA[IEC62601]) in
   the area of industrial wireless network currently.

   Industrial backhaul network is a transition network, which combines
   industrial field network with higher level network to achieve the
   goal of interconnection. It mainly solves the problem of access of
   industrial field network data to higher level network. Industrial
   field network is generally limited to a specific region, such as a
   plant. Through this network, transaction data of industrial field
   network can be transferred to internet or other industrial field
   networks. Industrial backhaul network is a medium-sized network,
   which can cover from a few kilometers to tens of kilometers. The
   major technology of industrial wireless backhaul network consists of
   Wi-Fi, WiMAX or LET.

   In order to adapt the presentation and development of industry 4.0,
   which is aimed to elevate the level of manufacturing, industrial
   field network should not be confined to a plant network only.
   Therefore, it is necessary to introduce the technology of industrial
   backhaul network to break the restrictions of interconnection
   between different networks, and to form a mixed network of
   industrial field network and backhaul network. Figure 1 indicates a
   typical network architecture of the mixed network. It is a type of
   deterministic network, and had been illustrated about use cases and
   architecture in the drafts proposed by DetNet Workgroup of IETF of
   draft-bas-usecase-detnet-02 and draft-finn-detnet-architecture-04.

Wang, et al.          Expires November 12, 2017               [Page 3]
 Internet-Draft           DetNet Joint Scheduling              May 2017

                  +-----------------------------------+
                  |                                   |
                  |                                   |
                  |           Backhaul network        |
                  |                                   |
                  |                                   |
                  +-----------------------------------+
                           /                  \
                          /                    \
   +-------------------------------+  +-------------------------------+
   |                               |  |                               |
   |                               |  |                               |
   |         Field network         |  |         Field network         |
   |                               |  |                               |
   |                               |  |                               |
   +-------------------------------+  +-------------------------------+

            Figure 1. Typical industrial field-backhaul network

   In this mixed network architecture of industrial field network and
   backhaul network, field network is made up of ISA100.11a, which is
   industrial wireless sensor network protocol. In the former network,
   a node deployed in a plant can communicate with a node deployed in
   another plant through a backhaul network.

2. Deterministic Industrial Field-Backhaul Network Requirement

   The draft of draft-finn-detnet-problem-statement put forward by
   DetNet Workgroup of IETF had described the requirement of
   deterministic network and deterministic scheduling partially.
   Because industrial field network directly faces the monitoring of
   industrial process, it is a difference between industrial field
   network data and general network data. Industrial field network has
   high demands about the deterministic delay bounds. It will affect
   the productivity, and even generate industrial accidents, when there
   are high packet loss and latency in a field network. For instance,
   real-time monitoring of level measurement and control are required
   to avoid overfilling of oil tanks that may lead to serious economic
   loss and environmental threats.

   So, it is needed that a deterministic joint scheduling method can
   guarantee the determination of network data in such a new network
   architecture.

Wang, et al.          Expires November 12, 2017               [Page 4]
 Internet-Draft           DetNet Joint Scheduling              May 2017

3. Deterministic Industrial Field-Backhaul Network Joint Scheduling Key
   Technology

3.1. End-to-end Network Data Stream

   In an industrial field network, end-to-end network data stream
   indicates a complete transmission path that a source device node
   transfers to a destination device node (common node or gateway).
   While in an industrial field-backhaul network, it indicates a
   complete transmission path that a field network source device node
   transfers through an industrial backhaul to another field network
   destination device node.

   Industrial field-backhaul network data stream have following
   features:

   o Period. Every data stream in network generates period data.

   o Deterministic. Every data stream in network has a deadline,
      network scheduling should ensure every data stream arrive at
      destination node before its deadline.

   o Sequential. A path of an end-to-end network data stream are made
      up of every two sequential node transmission link. In the process
      of scheduling, it must be scheduled by the order of sequence of
      links in the path.

   o Priority. Every end-to-end network data stream has a priority,
      when the data streams with different priorities occur collisions,
      the data stream with lower priority should be delayed.

3.2. Network Communication Resource

   In the deterministic industrial field networks with backhaul network
   architecture, schedulable network communication resources are time
   slot, channel and link. If the backhaul network is based on the SDN
   architecture, then the SDN controller could schedule the bandwidth
   and cache of switch. Therefore, bandwidth and cache resources can be
   included in schedulable network communication resources.

   o Time slot. Time slot is the basic unit in the TDMA based network
      communications. The length of time slots is settled and is same
      in the entire network. Only one packet and its ACK can be
      transmitted in one time slot.

Wang, et al.          Expires November 12, 2017               [Page 5]
 Internet-Draft           DetNet Joint Scheduling              May 2017

   o Channel. In order to increase network throughput, industrial
      field network standards provide a number of channels of different
      frequencies. If the links do not interfere with each other, then
      we can use different channels to transmit simultaneously.

   o Link. Link refers to a direct communication link between one node
      and another and no intermediate switching nodes. The network data
      stream is composed of a lot of links. The devices in the
      industrial field network devices are half-duplex, so the links in
      the industrial field network are unidirectional.

3.3. Network Time Slot Scheduling

   In TDMA-based industrial field network, time is divided into time
   slots of the same length. One transmission can be conducted in each
   time slot and the links using different channels to transmit if they
   do not interfere each other.

   In the time slot scheduling process, it will cause link collision
   when a node arranged to transmit and receive simultaneously, and it
   will cause channel collision when the same channel is used within a
   certain range. AS shown in figure 2, the network time slot
   scheduling process should avoid such collisions.

        +---+   +---+   +---+           +---+   +---+   +---+   +---+
        | A |-->| B |-->| C |           | A |-->| B |   | C |-->| D |
        +---+   +---+   +---+           +---+   +---+   +---+   +---+

         +---------+------------+         +---------+------------+
         |Time slot| Time slot 0|         |Time slot| Time slot 0|
         +---------+------------+         +---------+------------+
         |Channel 0|   A->B     |         |Channel 0|   A->B     |
         +---------+------------+         |         |   C->D     |
         |Channel 1|   B->C     |         +---------+------------+
         +---------+------------+
              Figure 2. Link Collision & Channel Collision

4. Joint Real-Time Scheduling Methods for Deterministic Industrial
   Field-Backhaul Network

   Joint real-time scheduling methods for deterministic industrial
   field/backhaul networks, which cross networks, are intend to solve
   the deterministic problem of industrial field /backhaul networks.
   Since the current network infrastructure imports backhaul network,
   the deterministic scheduling algorithm need to collaborate with
   backhaul network to conduct joint scheduling to ensure data
   certainty. The proposal put forward the following solutions.

Wang, et al.          Expires November 12, 2017               [Page 6]
 Internet-Draft           DetNet Joint Scheduling              May 2017

4.1. Time-Slotted Industrial Backhaul Networks

   In order to ensure determinism, industrial field networks utilize
   TDMA to make the network time-slotted. If the industrial backhaul
   network can also be time-slotted, then the deterministic scheduling
   algorithm can jointly schedule with minor alterations. Industrial
   backhaul network can be built with a variety of network standards
   such as Wi-Fi, WiMAX, LTE and so on. But in consideration of the
   high cost and poor feasibility of time-slotted WiMAX and LTE, we
   assume that the IEEE802.11 can be time-slotted. Wi-Fi network has
   various networking modes, such as peer to peer networking mode,
   point to multi-point networking mode and the relay network mode.
   Here we consider the hierarchical network constructed in point to
   multi-point networking mode, as shown in Figure 3.

               +----------------------------------------+
               |                                        |
               |               +--------+               |
               |       +-------| Head AP|-------+       |
               |       |       +--------+       |       |
               |       |                        |       |
               |   +--------+              +--------+   |
           +---+---|   AP1  |              |   AP2  |---+---+
           |   |   +--------+              +--------+   |   |
           |   +----------------------------------------+   |
           |                                                |
 +---------------------------------++----------------------------------+
 |ISA100.11a field wireless network||ISA100.11a field wireless network |
 +---------------------------------++----------------------------------+

          Figure 3. Industrial Backhaul Network consisting of WIFI

   Although IEEE802.11 also supports 13 channels, but the AP was not
   free to switch channels, which means that the AP cannot use a
   channel in the current time slot and use another channel the next
   time slot. However, we assume that the network architecture, the
   following points AP under head AP, which are AP1 and AP2 in FIG 1,
   can transmit packets simultaneously as long as their transmission
   task do not contain the same AP, i.e. head AP. For example, when a
   data stream of field network is transmitting packets to AP1 in a
   time slot, AP2 is able to receive packets from head AP, or send
   packets to field network in the same time slot. Therefore, the
   backhaul network constructed with wireless APs can be considered as
   a single-channel linear network, which is shown in Figure 4.

Wang, et al.          Expires November 12, 2017               [Page 7]
 Internet-Draft           DetNet Joint Scheduling              May 2017

+---------+    +--------+     +--------+    +--------+    +---------+
| Gateway |--> |   AP   |-->  |   AP   |--> |   AP   |--> | Gateway |
+---------+    +--------+     +--------+    +--------+    +---------+

                 Figure 4. A single-channel linear network

   Therefore, the data stream in industrial field/ backhaul network can
   be deemed to be equivalent to the data stream in field network, only
   that every piece of data streams need to go through the linear
   network consisting of wireless APs. So the scheduling process is
   proposed as follows:

   1. Abstract end to end data stream in the entire network, and
      initialize a different priority for each stream.

   2. Establish the delay model of network data stream. If collisions
      happened between different priority data stream, the low-priority
      data stream will be delayed by high-priority data stream, so a
      model can be built under the worst circumstances that the low-
      priority data streams impacted by higher priority data streams.

   3. Estimate the network schedulability. A data stream is schedulable
      if the minimum time for the data stream to complete transmission,
      plus the worst delay time caused by higher priority data streams,
      is less than or equal to deadline, In the current priority
      allocation scheme, if each data stream is schedulable, the
      network can be considered as schedulable. If the data stream
      cannot be scheduled, then change the priority allocation scheme
      and estimate again until a corresponding scheme is found.

   4. Allocate time slot and channel for every data stream. Traverse
      data streams according to their priorities, and each data stream
      should allocate the next link that is about to be released in
      each time slot to the greatest extent. According to the rule that
      low-priority data streams should give way to high-priority data
      streams, the spare channels can be utilized if there is no
      collision. However, if collisions happened between data streams
      of different priority, then the lower-priority data stream should
      be placed in the next time slot until there are no unallocated
      higher priority data streams. Follow these rules until the whole
      network scheduling is completed.

   The scheduling process is shown in Figure 5:

Wang, et al.          Expires November 12, 2017               [Page 8]
 Internet-Draft           DetNet Joint Scheduling              May 2017

          +----------+
          |   Begin  |
          +----------+
               |
               |
   +---------------------------+
   |  Initial the priority of  |
   |      each data stream     |
   +---------------------------+
               |<--------------------------------------+
               |                                       |
     +--------------------+            +------------------------------+
    /  Traverse every data \     no    | If the data stream cannot be |
   / stream and estimate the\--------->| scheduled, then change the   |
   \ schedulablity according/          | priority allocation scheme   |
    \   to delay model     /           | and estimate again           |
     +--------------------+            +------------------------------+
               |
               |yes
   +-----------------------------------+
   | Traverse data streams according to|
   | their priority, each data stream  |
   | should allocate the next link that|
   | is about to be released in each   |
   | time slot to the greatest extent  |
   +-----------------------------------+
               |
               |
   +-----------------------------------+
   | The spare channels can be utilized|
   | if there is no collision. If      |
   | collisions happened, then the     |
   | lower priority data stream should |
   | be placed in the next time slot   |
   +-----------------------------------+
               |
               |
           +-------+
           |  End  |
           +-------+
    Figure 5. Scheduling of times-slotted industrial backhaul network

Wang, et al.          Expires November 12, 2017               [Page 9]
 Internet-Draft           DetNet Joint Scheduling              May 2017

   Further, if the backhaul network can support TDMA mechanism like the
   industrial field network completely, the deterministic scheduling
   methods in field network can be applied in industrial field/backhaul
   networks.

4.2. Consider Industrial Backhaul Network as a Black Box

   In order to solve the deterministic problem of industrial backhaul
   network, industrial backhaul can be deemed as white box to conduct
   fine controls through inner mechanism. While it can also be regarded
   as a black box so that we can only consider its delay impacts and
   ignore its internal details.

   When the packet goes through the industrial backhaul network, we can
   give it a timestamp at the application layer and read it after the
   transmission completed. Then delay caused by the backhaul network
   can be figured out and a fitting curve of delay can be worked out by
   collecting large amount of data. It has been verified experimentally
   that the delay is concentrated in a numerical range despite its
   randomness. Therefore, we can get the approximate delay of packets
   caused by the industrial backhaul network.

   After that, a few of scheduling paths of different priority can be
   implemented in the industrial field network. A main scheduling path
   can be configured according to the average delay of the backhaul
   network. And some redundant paths should be pre-configured in case
   the delay of the main path is too high.

   The scheduling process of industrial field/backhaul network can be
   divided into three periods, as shown in Figure 6:

+--------------------+  +-----------------+  +------------------------+
|   Scheduling of    |  |    Delay of     |  |      Scheduling of     |
|source field subnet |->| backhaul network|->|destination field subnet|
|   (deterministic)  |  |(indeterministic)|  |( deterministic dynamic)|
+--------------------+  +-----------------+  +------------------------+
        Period 1              Period 2                Period 3

                   Figure 6. Three periods of scheduling

   In source field subnet we can apply the deterministic scheduling
   algorithm of field network to conduct deterministic polymerization
   and get the time spent by each data stream to go through the source
   subnet. Then the data stream goes through the backhaul network,
   which is a black box and it will cause indeterministic delay which
   is in a numerical range. When the data stream comes out the backhaul
   network, the timestamp should be parsed. If the deadline is missed,

Wang, et al.          Expires November 12, 2017              [Page 10]
 Internet-Draft           DetNet Joint Scheduling              May 2017

   it indicates that the packet has gone through poor network and need
   to be retransmitted. If there is time left, scheduling path can be
   dynamically selected at downward gateway to get the schedulability
   of the end to end data stream.

4.3. Ignore the Delay of Industrial Backhaul Network

   Since the field network is slow-speed (250 KB/s), while industrial
   backhaul network is a high-speed, if the industrial backhaul
   networks adopt IEEE802.11, gigabit wireless routers supporting
   IEEE802.11 ac can make the delay of industrial backhaul network
   quite small. As a result, the joint deterministic scheduling of the
   entire network only needs to cover the field network that is located
   at the ends of the backhaul network.

4.4. Build Delay Model of Industrial Backhaul Network

   If industrial backhaul network is built with IEEE802.11, the network
   access delay test model under IEEE802.11 DCF mode can be established
   by using Markov chain or queuing theory. At the same time, the model
   under IEEE802.11 PCF mode can be established based on queuing theory.

   Therefore, the field network only need to build the delay model of
   backhaul network that follows one delay model, then the total
   transmission scheduling delay will follow certain regularity. The
   total transmission delay will meet delay requirements with specified
   probability by scheduling, in other words, the unsuccessfulness of
   scheduling is acceptable, but the scheduling success rate should be
   in a range of 90% ~ 95%.

5. Security Considerations

6. IANA Considerations

   This memo includes no request to IANA.

7. References

7.1. Normative References

7.2. Informative References

[IEC62734]
           ISA/IEC, "ISA100.11a, Wireless Systems for Automation,
           also IEC 62734", 2011, <http://www.isa100wci.org/enUS/
           Documents/PDF/3405-ISA100-WirelessSystems-Future-brochWEB-
           ETSI.aspx>.

Wang, et al.          Expires November 12, 2017              [Page 11]
 Internet-Draft           DetNet Joint Scheduling              May 2017

[IEC62591]
           IEC, "Industrial Communication Networks -
           Wireless Communication Network and Communication Profiles
           - WirelessHART - IEC 62591", 2010,
           <https://webstore.iec.ch/p-
           preview/info_iec62591%7Bed1.0%7Den.pdf>

[IEC62601]
           IEC, "Industrial networks - Wireless communication network
           and communication profiles - WIA-PA - IEC 62601", 2015, <
           https://webstore.iec.ch/preview/info_iec62601%7Bed2.0%7Db.pdf>

[I-D.finn-detnet-problem-statement]
           Finn, N. and P. Thubert, "Deterministic Networking Problem
           Statement", draft-finn-detnet-problem-statement-04 (work in
           progress), October 2015.

[I-D.finn-detnet-architecture]
           Finn, N., Thubert, P., and M. Teener, "Deterministic
           Networking Architecture", draft-finn-detnetarchitecture-03
           (work in progress), March 2016.

[I-D.bas-usecase-detnet]
           Kaneko, Y., Toshiba and Das, S, "Building Automation Use
           Cases and Requirements for Deterministic Networking", draft-
           bas-usecase-detnet-00 (work in progress), April 2016.

Wang, et al.          Expires November 12, 2017              [Page 12]
 Internet-Draft           DetNet Joint Scheduling              May 2017

Authors' Addresses

   Heng Wang
   Chongqing University of Posts and Telecommunications
   2 Chongwen Road
   Chongqing, 400065
   China

   Phone: (86)-23-6248-7845
   Email: wangheng@cqupt.edu.cn

   Ping Wang
   Chongqing University of Posts and Telecommunications
   2 Chongwen Road
   Chongqing, 400065
   China

   Phone: (86)-23-6246-1061
   Email: wangping@cqupt.edu.cn

   Hang Yang
   Chongqing University of Posts and Telecommunications
   2 Chongwen Road
   Chongqing, 400065
   China

   Phone: (86)-23-6246-1061
   Email: 18716322620@163.com

Wang, et al.          Expires November 12, 2017              [Page 13]